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The Center. 



The Center for Social Organization of Schools has two 
primary objectives: to develop a scientific knowledge of 
how schools affect their students, and to use this knowledge 
to develop better school practices and organization. 

The Center works through three research programs to 
achieve its objectives. 

The School Organization Program investigates how school 
and classroom organization af f «3cts student learning and 
other outcomes. Current studies focus on parental involve- 
ment , microcomputers , use of time in schools , cooperative 
learning, and other organizational factors. The Education 
and Work Program examines the relationship between schooling 
and students' later-life occupational and educational suc- 
cess. Current projects include studies of the competencies 
required in the workplace, the sources of trainings and 
experience that lead to employment , college students' maj or 
field choices, and employment of urban minority youth. The 
Delinquency and School Environments Program researches the 
problem of crime, violence, vandalism, and disorder in 
schools and the role that schools play in delinquency. 
Ongoing studies address the need to develop a strong theory 
of delinquent behavior while examining school effects on 
delinquency and evaluating delinquency prevention programs 
in and outside of schools. 

The Center also supports a Fellowships in Education 
Research program that provides opportunities for talented 
young researchers to conduct and publish significant 
research and encourages the participation of women and 
minorities in research on education. 

This report, prepared by the Delinquency and School Envi- 
ronments Program, describes further interim results of the 
program's national evaluation of the Office of Juvenile Jus- 
tice and Delinquency Prevention's (OJJDP's) Alternative Edu- 
cation program. The first interim results were reported in 
CSOS Report No. 325, April 1982. 
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Compton Action Alternative School: Second Interim Report 



B. K. Daniels and G. D. Gottfredson 



Abstract 



The Compton Action Alternative School (formerly the Compton Action 
Center for Youth Development, CACYD) is a delinquency prevention demons- 
tration project sponsored by the Office for Juvenile Justice and Delin- 
quency Prevention (OJJDP) as part of the OJJDP Program in Delinquency 
Prevention through Alternative Education. CACYD has evolved over the 
course of its first two years in ways that appear to have • strengthened 
it as a delinquency prevention project. Interim results based on stu- 
dent self-report suggest that the project has been remarkably effective 
in altering a number of student character istics that delinquency preven- 
tion theory implies must be altered to prevent del inquent behavior , and 
student self-reports of delinquent tehavior are significantly lower than 
the self-reports of a control group. The self-report data must be 
interpreted with caution, however, because of some evidence of differen- 
tial validity for treatment and control group members. Problems with 
the retrieval of some archival data on official delinquency and other 
outcomes limit the assessment reported here. New data have very 
recently become available to strengthen the present analyses, and this 
report should be regarded as tentative. It is subject to revision as 
new results emerge. 



The Compton Action Alternative 
School (CACYD) has gone through a 
developmental sequence that paral- 
lels the development its clients 
must go through. That is, just as 
its clients must first come to see 
themselves as students and then 
learn the behaviors associated with 
being students, CACYD first had to 
establish its identity as a school • 
Two years ago the CACYD was a youth 
and family counsel ing center cal led 
the Compton Action Center for Youth 
Development (hence the acronym) with 
a mini-school component providing 
only short-term services for Compton 
Unified School District (CUSD) stu- 
dents. For the first year it con- 
centrated its energies on becoming a 
school — and it attempted to teach 
its clients that one of the things a 
student does is come to school. In 



its second year , the Center placed 
emphasis on refining its interven- 
tions designed to achieve affective 
and behavioral objectives, and it 
tried to teach its clients the 
behaviors associated with being stu- 
dents , to have conf idence in their 
abilities in the school environment, 
and to have conf idence in the 
staffs concern about their educa- 
tional progress. ^ 



This second interim report sup- 
plements an earlier report (Daniels, 
1982) which should be consulted for 
a basic overview of this project and 
an account of earlier activities. 

1. In the third year (1982-83) the 
Center is emphasizing academics. 
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Programmatic Changes in the Second 
Year 

The Center's clients are youths 
who have not adapted well in a trad- 
itional school setting. In all 
likelihood they were regarded by 
their teachers in the public schools 
as "slow" or "troublesome," student 
characteristics generally regarded 
as undesirable in public education. 
The Center's approach to these 
clients is to tell them that they 
can learn . For those who are slow 
learners, instructors provide as 
much time and repetition of the 
material as it takes for them to 
learn what is being taught. They 
also aim to provide enough opportu- 
nities for them to succeed that they 
can begin to realize they can learn. 
This process is intended to lead 
students to come to understand that 
it is OK go be slow; and that it is 
important that they can learn 
nevertheless. To promote this 
learning, CACYD implements its con- 
cept of personalized education ^ 
This term means individualized edu- 
cation with educational aims broadly 
construed, i.e., not limited to aca- 
demic concerns but also to the 
affective and behavioral domains. 
The Center assumes that different 
motivators, teaching styles, and 
materials may be most effective for 
different students ; and it assumes 
that different interpersonal styles 
may be effective for different stu- 
dents.^ 



2 . Unlike the first and second 
years, each student took a battery 
of tests in the third year (the 
Gates and California Achievement 
Test ) . These tests were used to 
place students in achievement 
groups, a procedure also not used in 
the first and second year. 



As a part of the personalized 
educational approach. Individual ized 
Education Plans (lEPs) were prepared 
for all students by each instructor. 
The intention was for the plans to 
include an assessment of each stu- 
dent's skills at entry or the begin- 
ning of the school year, individual 
academ ic and behav ior al ob j ect ive s , 
and plans to meet those obj ectives. 
The plans were to be revised and 
updated every five or six weeks* At 
each periodic update, the plans were 
to specify whether the" student was 
meeting the objectives and to update 
th e ob j e c t ive s or th e me th od s and 
materials to be used in the coming 
period to assist the student in 
achiev ing the obj ect ive s . The 
instructor was to inform each stu- 
dent of the plan. A review of a 
sample of these plans showed that 
the format did not include space for 
behavioral obj ectives and that aca- 
demic obj ectives ^sought_and_plans _to 
achieve them often needed to be more 
precise and detailed. Most of these 
plans were updated regularly as 
planned . Administrators are aware 
that additional attention is needed 
to improve the implementation of 
these lEPs. A revised form which is 
somewhat more specific in the infor- 
mation requested was introduced near 
the end of the second semester for 
use in the third year. 

Academic credit was granted 
through the Compton Unified School 
District. The curriculum included 
(a) reading, (b) English, 

(c) mathematics, (d) spcial studies, 
and (e) computer-assisted learning. 
Whereas in the first year survival 
skills were included as modules in 
each class, in the second year a 
separate class was established that 
exposed students to tasks such as 
the completion of job appl ication 
blanks , letter writing, completion 
of rental appl ication forms, and 
understanding gas and electric 
bills. Unlike CUSD schools, where 
classes extend into the af ternoon. 
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at the Center classes are held until 
1:30 p.m., including half an hour 
for lunch. 

Interviews with staff, and direct 
observation, indicate that most 
instructors attempt to individualize 
instruction, but additional work may 
be needed to achieve individual iza- 
tion in all classrooms. Some staff 
have also expressed a desire for 
assistance in improving their teach- 
ing methods, and a desire to make 
the curriculum more experiential. 

The Typical School Day 

The typical day starts off with a 
half -hour "morning meeting," or 
assembly. This meeting was initi- 
ated as an incentive to get students 
to school on time (and is also 
viewed as a vehicle for promoting 
student participation and capacities 
to plan and identify problems). 
Staff members who rotate through 
this responsibility share responsi- 
bility for running the morning meet- 
ing with two students. It is gener- 
ally held as an open forum, where 
students debate, make presentations, 
and plan for daily events. Issues 
to be taken up by the student 
government are developed in the 
morning meetings. Students planned 
and implemented several activities 
this year that were based on ideas 
that arose in these morning meet- 
ings. The largest such undertakings 
were a prom and graduation which 
they initiated and helped to plan, 
and a week of debates loiown as "The 
Great Debate." All students and 
staff members participated in these 
debates . The topics debated were 
"Decriminalizing Marijuana," "Pre- 
Marital Sex," and "The Criminal 



3 • In the third year a student 
government has assumed the responsi- 
bility for running the morining 
meeting. 



Justice System." 

After the morning meeting, stu- 
dents begin their academic course 
work. Some , for instance , began 
work on the Plato (computerized 
instruction) system. The on-line 
Plato system used in the first year 
was replaced with micro-Plato which 
uses discs . This smal ler system 
offers fewer programs than the lar- 
ger system, but eliminated problems 
with phone lines that were experi- 
enced in the first year. A teacher 
supervises this activity, supple- 
menting work on the computer with 
classroom instruction and practice 
, in work books. Students spend time 
working with PLATO every day. 

Teaching Approach 

No set formula for instruction is 
provided for staff (three full-time 
instructors , two half-time instruc- 
tors, and a teaching aide), but gen- 
eral guidance has been provided. 
Four elements of effective teaching 
have been communicated to the staff 
by the project director: 
(a) "direct instruction, " (b) time 
to exercise the skills to be tran- 
smitted , (c ) time spent in review 
and exercise of the skills, and 
(d) testing to check for mastery. 
In addition, several training ses- 
sions for the staff in the Kawaida 
method were held. The method 
appears to be directed to enhancing 
self-esteem, identity , planf ulness, 
and responsibil ity on the part of 
students by encouraging instructors 
to provide opportunities for the 
mastery of skills, recognition and 
reinforcement of success, and accep- 
tance. The method was developed by 
Gary and Maye Beale who drew heavily 
on an impublished dissertation by 
Joseph White. The materials we have 
seen seem to draw heavily on the 
popular writings of Glasser (1969; 
1975). 
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Additional Educational Features 

Several kinds of activities occur 
either periodically or occasionally 
to supplement the day-to-day rout-- 
ine : 

Career dav # The Center^ s commu-- 
nity advisory committee helped spon^ 
sor a career day in which a number 
of professionals, including a police 
officer, came to the school to dis^ 
cuss their work with the stu\nts. 
The staff felt an important feature 
of this career day was that it was 
the first time that many profession-- 
als had an opportunity to talk to 
so-called "bad" students in a posi-- 
tive setting, and vice versa. 

Inf onnal student-staff intej- .c-- 
t ion . As an ap pr o a ch to pr ov id i n g 
rewarding social interactions bet-- 
ween students and teachers that are 
not provided in traditional school 
structures, staff are expected to 
spend time with students by engaging 
in non-academic interaction. An 
example of this form of interaction 
is the potluck lunches and dinners 
involving parents and students. 

Fonnal interaction * Fiel d trips 
to an amusement park, a museum, some 
local colleges and universities, and 
to sports and entertainment events 
were carried out. 

A partial token economy . Near 
the end of the second year, the Cen- 
ter began to implement a partial 
token economy syst^ni^ Students 
could earn tokens for class atten- 
dance and participation that could 
be redeemed in a recreation area to 
play pool or engage in other activi^ 
ties. 

Student government * A student 
government was formed with eight 
elected members . The government 
initiated ad hoc committees involv- 
ing other students to plan and 



impl^ent activities such as a fund- 
raiset- for a field trip and the 
revision of t-i^l^^ that govern stu- 
dent behavior, 

Malntenanc ft^ejgi^aloyment activity . 
As one way to li^lp solve a graffiti 
probl ^ dur ing the second semester , 
two students rotated through jobs to 
keep the Center: clean. Students 
must sign in, ^tid they are expected 
to learn conveti^ional work habits. 
They are paid fot two hours work and 
Volunteer for ^Vo hours of work. 
The students holding these jobs are 
recomtnended by ^he staff, and this 
Work assignment is sometimes used as 
„a reward for ^ctiieving a goal or as 
an incentive improve when a stu- 
dent is not achieving goals. 

Mentors . Attempts were made 
again to impl^tn^nt the mentor pro- 
gram that was Unsuccessful in the 
first year.^ Ag^^^, _this_ attempt was 
not totally " S^QceSsful, although 
sotne community involvement was 
achieved. One IJCLA student con- 
ducted two proj ^cts with students, 
but attempts t:0 t-^cruit mentors from 
other closer colleges failed, par- 
tially because ^? cutbacks in work 
study funds and Partially because of 
a reported la^K of interest on the 
Part of the college students. Prior 
to cuts in C^TA funding, some CETA 
wofke^rs were ^nployed by the Center 
to work with ^ti^dents in the mentor 
role- Although students apparently 
related bette^ CETA workers than 
to College s^i^^^nts, and students 
undertook proj^^ts such as con- 
structing East^^ baskets for disa- 
bled children, funding cuts pre- 
vented this activity from 
Continuing. 

Task or ien h^^on and_ atmosphere . 
Two additional features of the edu- 
cational envi^^omuent were salient in 
students^ responses to an interview 
question. W^en asked what they 
Would tell soti^eOTie about the Center 
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if asked to describe it, the most 
common response was that students 
have to get their work done every- 
day. Student responses to interview 
questions also emphasized that they 
felt good about themselves and the 
staf f , that they f el t relaxed , as 
though they could learn, and that 
they were sure the staff was con- 
cerned about them learning. 

Parent and Community Activities 

Parents . Fprmal parent training 
was planned, and six workshops were 
held. The topics covered were "Kow 
the CUSD Functions," "A Candidates 
Forum for Persons Running for the 
CUSD Board of Trustees," "Parenting 
Adolescents, " "Preventing Drug and 
Alcohol Use by Teens," "Gang-O- 
riented Youth," and a "Family After- 
noon." The last of these was a 
pot-luck meal shared by parents, 
students , and staff at which the 
workshop series was reviewed. The 
response on the part of participants 
was reportedly favorable, but only 
six or seven parents participated in 
all the workshops. (The first two 
workshops drew approximately 35 par- 
ticipants.) Considerable parent 
contact and involvement was achieved 
in other ways. This included commu- 
nication with parents when students 
were absent , a parent council , and 
parent participation in Center 
activities . 

Parents were contacted by tele- 
phone when students were absent. 
Although this is a difficult activ 
ity to conduct , and although many 
students of the Center live in non- 
traditional families, this was a 
regular and recurring activity. 

A seven-member parent council 
held meetings about once per month 
during the year. This council was 
primarily a discussion and support 
group for parents. 



that parents seem to respond better 
to requests to participate in school 
activities than to requests to 
attend meetings. Accordingly, 
parents have been encouraged to 
engage in activities such as the 
career day and a car wash. This 
appears to be a fruitful approach, 
and suggests that parent involve- 
ment in school activities might be 
fostered by providing them with 
meaningful roles in the conduct of a 
school's activities. ■ An attempt has 
also been made to *get parents 
involved in the student contracts 
involving educational plans, but 
this has not met with great success 
nor been attempted as regularly as 
had been envisioned. 

Community . A 13-member community 
advisory council has been formed. 
It includes a Compton attorney, 
business persons, representatives of 
the probation and narcotics parole 
departments, a clergyman, a Congres- 
sional staffer, and two students. 
We have not interviewed any of these 
individuals about their roles , but 
the function of the committee is 
reportedly to give advice , to pro- 
vide an indication of community sen- 
timent, and to help develop good 
will for the Center and help it to 
solve problems. 

The Center encouraged informal 
community participation in other 
ways. For example, a community mem- 
ber dropped in, volunteered to 
tutor , and did so for a number of 
weeks. And a local liquor store 
owner contributed to and partici- 
pated in student-sponsored activi- 
ties. 

Other Project Improvements 



S t af f tur nov e r wa s high in th e 
Center's first year. Staffing was 
stable in the second project year. 
One teaching aide was dismissed 
because she was associated with a 



The project manager has observed 
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gang, and the Center wanted to be 
perceived as neutral with respect to 
competing gangs in the community. 
Quarterly staffing reports show 
that, exclusive of CETA workers, the 
project had 8.4 full-time equivalent 
workers through May, 1982, with no 
turnover during the year until that 
time. Hie project staff decreased 
by one person in May, one full-time 
and one 70% t ime ^worker June , and 
one person (the principal ) in 
August, 1982. Two full-time CETA 
workers were lost in May when their 
time allotment in the program 
expired. . 

The Center has engaged in plan- 
ning using the PDE method more fully 
in its second year of operation. It 
has developed its activities accord- 
ing to a more fully specified plan, 
and it has involved more staff mem- 
bers regularly in the formal plan- 
ning process. Although all staff 
members do not view the PDE method 
as equally useful, nor do all use it 
in equal amounts, most staff members 
know about the Center's PDE plan and 
were able to explain how planning 
following this method works. 

Difficulties with on-site evalua- 
tion activities experienced in the 
first year have ceased, and the on- 
site evaluator is doing an excellent 
job of implementing evaluation-re- 
lated activities. 

Progress Towards Institutionaliza- 
tion 

We define the institutional iza- 
tion of a program as a condition in 
which no special effort is required 
to maintain its operation as a part 
of the status quo, and in which con- 
siderable effort would be required 
t o end or mo d i f y th e pr ogr am . By 
this definition, the Center is 
clearly not yet institutionalized. 
It appears to be making progress 
towards institutionalization in the 
sense that obstacles to 



its operation are decreasing somew- 
hat and it is acquiring some of the 
trappings of an institution. 

The only noticeable opposition to 
the Center's activities has come 
from the police who feel that addi- 
tional security is required for the 
Center to remain in operation in its 
present location. (There was one 
serious gang intrusion into the cen- 
ter this year. ) The Center's advi- 
sory committee has been of some help 
in interacting with " the police 
department. In addition, the Cen- 
ter's participation together with 
the pol ice in the District's Clean 
„ Sweep Program has increased respect 
for the Center on the part of the 
police, as well as other community 
entitites. One CUSD school princi- 
pal has shown some opposition, but 
this is not considered serious. 

In the second _ye.ar, J:he_Center's 
clients were carried on the CUSD 
rolls and received lunches through a 
CUSD program. There are additional 
signs that the Center's relations 
with the district are becoming more 
positive. Project managers now know 
more key people in the district and 
the new acting superintendent (AS ) 
is supportive of the Center's 
efforts. An interview with the AS 
implies that he views CACYD very 
positively and sees the Center as 
working closely with his schools. 
The Center's manager is a member of 
his Blue Ribbon Committee on School 
Change (chairing its subcommittee on 
educational change ) , and has been 
active in conducting surveys in CUSD 
schools intended to be helpful in 
planning programs to prevent CUSD 
teachers from al ienating students. 
The Center's principal is a member 
of the district's discipl inary 
review committee. , The AS plans to 
seek CACYD assistance in implement- 
ing and monitoring CUSD's efforts to 
increase individualization in 
instruction. 
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The Center managers and the AS 
both see the Center as an entity 
that should operate outside the 
public school system. The Center is 
intended to have a role of its own 
but to operate in cooperation with 
the CUSD to accomplish some common 
goals. Although the CUSD has pro- 
vided some services and furniture 
for CACYD there are no plans for the 
district to provide any substantial 
financial support. The AS reports 
that he would help with funding if 
he had the funds , as it is he sees 
his role as helping the Center 
obtain outside funding. 

Decreasing funding through the 
OJJbP grant may influence the pro- 
ject's ability to operate. Further- 
more cuts in work-study funding for 
colleges in the area, and cuts in 
CETA, have decreased the availabil- 
ity of low-cost supplementary work- 
ers. The poor economic climate and 
decisions to reduce the Federal role 
in the support of social programs 
will almost certainly work to reduce 
the likelihood that the Center will 
achieve institutional ization. 



Interim Summative Evaluation 

Probl ems exper ienced in imple-/ 
menting the evaluation in the first 
project year have been resolved. 
Project personnel have zea- 
lously pursued evaluation activities 
with the result that the evaluation 
design is strong and has been well 
implemented . In al 1 80 student s 
participated in the Center during 
the 1981-82 school year. Of these 
48 (60%) were still in attendance at 
the end of the school year.^ Attri- 
tion is accounted for as follows: 



involuntary withdrawn (primarily 
incarcerations ) , 15% of partici- 
pants; left Compton, 12%; voluntary 
withdrawal (dropout), 4%; return to 
regular school , 4%. 

Of the 91 students for whom we 
have data who participated in CACYD 
in the first year, 41 (45%) returned 
to the Center at some time during 
the second year. Of these 41 retur- 
nees, 66% were still in attendance 
at the end of the second year, 22% 
were involuntary withdrawal s (pri- 
marily due to incarceration), 
5% left Compton, 5% voluntarily 
withdrew, and 2% returned to a regu- 
„lar school. This pattern of student 
attrition implies that few students 
are voluntarily dropping out of 
school , and the largest source of 
attrition is incarceration of stu- 
dents in this high-risk population. 

Methods 

Assignment to treatment and con - 
trol groups . Except for returning 
students (N = 41), students eligible 
for participation were assigned to 
treatment or to an untreated control 
group. Because of pressure from 
OJJDP to rapidly increase the number 
of participants in the school, how- 
ever, assignment was not straight- 
forward. Pressure to rapidly bring 
the number of students served up to 
an acceptable level required the 
following procedure: Between the 
start of the school year and 14 
December 1981 , a computer generated 
list of pseudorandom numbers was 
provided to the on-site evaluation 
coordinator. The list contained 
lines a random 80% of which carried 
a code for treatment condition and 
the remaining 20% a code for control 
condition. As youths became eligi- 
ble for the program, their names 
were alphabetized and written in 

proce- 
assign 
tment. 



4. Our best estimate of the percen- that order on the list. This 

tage of participants who persisted dure was i-^tended to randomly 

until the end of the 1980-81 school 80% of el t. s youths to trea 

year is 58%. 

-149- 



iMiil^f Hit fi^itlyiiM^ fiifitifiMI fegr 
tf«MMi«t tr WM f * to M fmahB 

!!•« fi« i»ttitii«iit w taM c«««ff« 

iMMl til ««t« MNllMllNitf M M MfttrMt«4l 

«wM««t iv«if> SMrlM liM rim 

scHMiiMMi Mi tP ymtht «Nir» 
•MMPMI M liMi CMtr*! ir««f. Bwr- 

md 44 wnrn «t«l9Mpi u «i«tr*l* 
iMijinwiBI «Mif4l«it«t Ci* itcis* 



««(«»t if iMy Hm p»«I 

fHf litr* ^fi{«t«4 mrly md 

MM §i*fm9 U tt tmt— MM«i«C«4 
Urn m *Mfli«ff muI t«t«f 

ifMMI 4liff«f«4« tit «f CTMtMMlt 

ft IMS wItMfiwit frm •d»9ol (.03)« 
MtA^e «Mt««t C*ei)* Atuctumit ^ 
itfinti tOiMt f^rm 

idM>» l CMttlttg tM, r«rmt«l Km- 

§t^m% 9tmtt—4»m CkM ii f«r- 
tUAlf nimt§nt wiU vftiMl mti^ 
f«9fh IMw&li^M Attt«M»y (.02), 

«trtf md Ut» «itiiaMi>£ »*rlod. 
ni« 4MNMt that mtttt Uni •••iga- 

fct «i|*Ml«i u iiff«r iM to the 

«tft tiMP Wflf« MMf^Mli U aidiKiM 
t« My itff«rMlt»f iM 10 iPftOitbt* 



A ilfftnrftM mo alto fooatf 
WCMM ttootsmt oai eontrol groupt 
00 Moktro of (olleo co«ucct btt- 
mmm 1$ ttpU^r IMO Md 22 Angutt 
Iftl <!.•.. eke yoar Jhflm •••igo- 
MOt)* Hm tr««tM«t groap (N « 38) 
ted • warn of .U polieo contaect in 
Uat yoar aod tlM control group (K • 
32) li^ a aaaa of .60 poliea con- 
tacta. ft-tataMsc itvdoBti bad ftvar 
prior pollco coatacta than control 
jrontka la both tha aarly and lata 
aaaipMMC pariodc, and tha diffor- 
aiica ia atatiatically algnif leant (a 
< .004) ttiMn aaaignatant pariod 
ia atatiatically bald constant. A 
doM OKaaiaatioa of tba diatribu- 
•tiooa iopliaa tbat tbia diff*renc« 
caa Mt ba acco«»tad for by outli- 
ara. For tba traataant group the 
wbar of prior polica contacts 
raogad fre» 0 to A, vitb 55. 3Z hav- 
ing no prior contacta in tba pariod 
auainad. For tba control group the 
nvnbar of , prior polica contacts 
rangad fron 0 to 7, «itb 47 Z having 
no prior contacta in tha pariod axa- 
■iaad* Iba traataiant and control 
gronpa difforad aoaavhat in athnic 
conpeaition bat tbia diffarenca 
appaara diia to a diffaranea in the 
coBpeaition of tba eligible pools in 
tbe aarly and late asaignaant per- 
iods. Tba treataant group vaa 78Z 
•lack* and tba control group was 91 Z 
black. Ibia difference is not sig- 
nificant vban aaaignnant period is 
atatiatically controlled. 

Wa do not knov to what to attri- 
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of arraits. It ia possible that 
baciuse a disproportionate manber of 
control youths entered the cxperi- 
want in the late assignment period, 
nore of their arresta were reflected 
in the data base because a longer 
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period of time had elapsed between 
the arrest and the time the files 
were searched. On the other hand, 
the difference was observed for 
early and late assignment periods* 
We will continue to explore this 
issue and conduct new searches of 
the police contact data base to 
check on this hypothesis. 

Analyses * To assess the effects 
of participation in the Compton pro- 
ject in the most direct way possi- 
ble, a comparison was made of stu- 
dents who were assigned to 
participate and the control group 
taking into account the assignment 
period during which they entered the 
exper iment • ^ Where deemed appr opr i- 
ate, statistical adjustments were 
also made for prior police record. 

Measures * A single instrument, 
the School Action Effectiveness 
Study Student Questionnaire was used 
to collect information on both 
treatment and control students* 



5. In technical terms, the approxi- 
mate equivalent of a 2 by 2 facto- 
rial design with unequal cells was 
accomplished using the SPSS ANOVA 
procedure with option 10 on release 
9 of that package (Nie, Hull, Jen- 
kins, Steinbrenner , & Bent, 1975; 
Hull & Nie,» 1981). This procedure 
involves a regression approach, 
where the statistical test for main 
effects involves the incremental 
contribution of each factor net of 
prior factors (in this case the 
assignment period during which a 
person entered the treatment or con- 
trol groups) and covariates (prior 
police record when appropriate) • 
The interaction effect (treatment by 
assignment period) is tested after 
main effects are entered into the 
model. If an interaction effect 
significant at the p " .10 level or 
less was found, data were analyzed 
separately by period. It was neces- 
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This instrument is described else- 
where (Gottf redson, Ogawa, Rickert, 
& Gottfredson, 1982), and readers 
should consult that description for 
an account of some of its psychome- 
tric properties. This questionnaire 
was administered by the on-site 
evaluation coordinator in small 
groups or individually as necessary. 
The administration was closely 
supervised and resulted in high res- 
ponse rates' for both treatment and 
control groups, and a low percentage 
of item non-response. Control group 
members were offered $5.00 for com- 
ing to a questionnaire administra- 
tion. A total of 74% of treatment 
^ and 87% of control students were 
administered the questionnaire. 

In addition, the on-site evalua- 
tion coordinator collected several 
other pieces of information from 
school and police records. This 
information included police records 
and school attendance and withdrawal 
information. The measures examined 



sary to search for a main effect of 
treatment only after adjusting for 
assignment period because assignment 
period was significantly associated 
with a number of outcomes even when 
treatment was "controlled" out. We 
regard this procedure as a conserva- 
tive test of treatment effects 
because, as a practical matter, the 
period during which students are 
assigned to treatment is associated 
with the duration of exposure to 
treatment. Thus, applying a statis- 
tical control for period of assign- 
ment tends to work against finding 
treatment effects if effect size is - 
associated with duration of expo- 
sure, as might be expected. Depen- 
dable differences associated with 
both treatment condition and per- 
iod of assignment have been assigned 
to period in the analyses. This is 
unfortunate , but protects the 
believability of the statistical 
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in this report are briefly described 
below: 

Withdrawal from school > Withdra- 
wal from school, either voluntary or 
involuntary, is coded 1; any other 
observed outcome (left Compton, 
returned to school , or persisted in 
CACYD) is coded 0. 

Any police contact . Any ev idence 
of police contact for any offense 
during the period 16 September 1981 
through 1 August 1982 according to a 
search of arrest records in the 
Hall of Records auto- 

mated information retrieval system 
is coded 1. No evidence of police 
contact during that period is coded 
0. Access to these records was 
gained through a court order. 

Number of alleged offenses . This 
is the total number of offenses 
alleged during the period 16 Septem- 
ber 1981 through 1 August 1982. 

Attachment to parents . This is a 
questionnaire-based measure of 
"attachment," i.e. feeling close and 
wanting to be 1 ike parents • It is 
intended to measure an element of 
Hirschi's (1969) social bond. 

Negative peer influence . This is 
a scale based on student reports 
that his or her friends get into 



conclusions drawn* 

Multivariate analysis of variance 
was precluded because of excessive 
missing data. Our inability to pro- 
tect the univariate tests by per- 
forming multivariate, analysis of 
variance first means that some dif- 
ferences may be expected by chance 
alone.^ We interpret ^.univariate dif- 
ferences nonetheless, and this len- 
iency seems to us to counterbalance 
the conservative handl ing of nonor- 
thogonality in the design. 



trouble, do not like school, etc. 

Attachmerjb to school . This is a 
scale based on student reports that 
they like the school . It is 
intended to measure an element of 
Hirschi's (1969) social bond. 

Belief in rules . This is a scale 
based on student reports that taking 
advantage of others and breaking 
rules or laws is not OK. 

Interpersonal competency . This 
is a questionnaire-based measure of 
psychological health based on Hol- 
land and Baird^s (1968) scale with 
..the same name. 

Positive self-concept . This 
scale measures students' self-esteem 
and conception of themselves as pro- 
social, law-abiding citizens. 

Self-reported delinquent behav- 
ior . This scale is based on 19 
items asking whether the respondent 
has committed various kinds of del- 
inquent behavior during the past 
year. 

Self-reported substance use . 
This is a subset of the previous 
scale, limited to the use of drugs, 
alcohol , and inhalants. 

Self -reported serious delinquent 
behavior . This is a subset of the 
self -report del inquent behavior 
scale limited to the most serious of 
the offenses in that scale. 

School punishment . This scale is 
based on students' reports that they 
have been punished in various ways 
in school for their recent school 
behavior. 

School rewards . This scale is 
based on students' reports that they 
have been rewarded in various ways 
in school for their recent school 
behavior. 
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Victimization * This scale 

measures recent in-school victimiza- 
tion experiences. 

Invalidity . This scale is 
intended to detect nonveridical res- 
ponding. Reporting that one reads 
several whole books every day and 
that it is easy to get along with 
nasty people earns a high score. 

School effort . This is a questi- 
onnaire-based measure of ef fort 
expended at school work. 

Practical knowledge . This is a 
7-item self-report measure of know- 
ledge of several life competencies, 
such as how to balance a check book, 
arrange a trip out of town, or apply 
for a job. 

Alienation . This is a self-re- 
port measure of feelings of 
estrangement from the social order. 

Self-reported grades . This sin- 
gle-item measure is self-explana- 
tory. 

Self-reported reading ability . 
This single-item measure is self-ex- 
planatory. 

Educational expectation . This is 
the response to a questionnaire item 
asking how far the person expects to 
go in school • 

Work for pay . This measure is 
scored "1" if the respondent 
reported that he or she worked for 
pay during the past week, and "0" if 
the respondent indicated that he or 
she did not. 

Regular job . This indicator is 
coded "1" if the respondent indi- 
cated that he or she holds a full- 
or part-time job, and it is coded 
"0" if the respondent indicates that 
he or she does not. 



Suspension . This is a 
self-report of suspension from 

school during the current (Spring) 
term. 

School non-attendance index . 

This is a two-item measure of class 
cutting and school cutting. 

Rebellious autonomy . This scale 
is intended to measure an immature 
attitude of autonomy* A high score 
is earned by reporting that it is no 
one's business how the respondent 
spends* his or her money, for exam- 
ple. 

Involvement . This is a question- 
naire-based measure of participation 
in some common school activities. 

Results 

Treatment-control group compari- 
sons are presented in Table 1 • Of 
25 comparisons based on question- 
naire measures, 18 significantly 
favor the treatment group. ^ One of 
these significant differences favors 
the control group. In one case a 
statistical interaction was found, 
meaning that the results differ 
depending on whether the youths were 
assigned to treatment or control 
conditions in the early or late 
assignment period* 



6. These significance tests are not 
independent • Indeed , some of them 
are largely redundant (specifically 
the three self-report delinquency 
measures, two of which are based on 
subsets of items in the overall mea- 
sure). Therefore the significance 
1 ev el s sh own in the tabl e shoul d be 
regarded as nominal rather than 
actual levels. Some of these signi- 
ficant differences may arise due to 
chance alone • 
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No significant difference occurs 
for any of the measures based on 
archival evidence shown in Table 1 
(withdrawals, police contact, number 
of police contacts, suspensions, or 
grades).^ The treatment and control 
groups were vastly different in 
officially reported suspensions, 
however. CACYD reported only six 
suspensions in the entire year. In 
contrast , 81 .2% of control group 
youths were suspended at least once 
according to school records. 
(Because of differences in data col- 
lection methods this result is not 
tabled, and we have not calculated a 
significance test, but none is 
necessary. The difference is strik- 
ing. ) 

A threat to interpret ability . 
The one significant difference 
favoring the control group presents 
an important threat to the interpre- 
tation of the questionnaire results. 
Treatment students scored higher 
than control youths on the invalid- 
ity scale, a measure of non-veridi- 
cal reporting. Accordingly addi- 
tional checks of the validity of the 
self-report measures were made sepa- 
rately for the treatment and control 
youths. For the control group, 
the sel f-repor ted del inquent behav- 
ior scale correlates .54 (N = 34, £ 
< .001) with the number of police 
contacts in the 81-82 period. For 
the treatment group, this correla- 
tion is -.01 (N = 23, n.s.).S This 



result impl ies that the credibil ity 
of the self-report del inquency mea- 
sure is^highly questionable. 

Several other checks on the val- 
idity of sel f-repor t for the treat- 
ment and control groups are not 
reassuring. These checks are sum- 
marized in Table 2. Percentage of 
days in attendance calculated from 
archival data submitted for CACYD 
participants for the final quarterly 
reporting period by the Center cor- 
relates .31 (N = 29, £ < .10) with 
the School Nonattendance index based 
on self-report. By comparison, 
total days of absence during the 
,year based on CUSD records corre- 
lates .23 (N = 41, n.s.) with the 
School Nonattendance index. Self- 
reported reading abil ity correlates 
-.07 (N = 29, n.s.) with scores on 
a reading test the project used to 
assign students to levels in the 
Plato s^stepi and_.05 (N = _24j ^with 
CACYD grades. No reading test was 
available for the control group, but 
self-reported reading ability corre- 
lates .16 (N = 36) with CUSD grades 
for this group. Self-reported 
grades correlates with grades 
according to official records -.21 
(N = 25) and .10 (N = 37) for treat- 
ment and control groups , respec- 
tively. Finally, CACYD reported no 
administrative removal s at al 1 dur- 
ing its last reporting period, but 
28% of CACYD participants (N = 32) 
reported having been suspended dur- 
ing the last school term. It is not 



7 . There is reason to bel ieve that 
some errors may have been made that 
led to an underestimate of the num*- 
ber of withdrawals for control group 
youths. We are continuing to 
explore the completeness of the data 
in this regard. We are unable to 
compare treatment and control groups 
on archival attendance data at this 
cime. We are also continuing 
attempts to explore these data. 

8. The N's vary due to missing 



data. Due primarily to selective 
non-response to these particular 
items , del inquent behav ior sel f-re- 
ports were available only for 59% of 
treatment students (N = 39) and 64% 
of control group members (N = 54) . 
This difference is nonsignificant. 
In contrast , pol ice records were 
available for almost all treatment 
(97.4%) and control group (98.1%) 
youths. 
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possible, of course to calculate a 
correlation when there is no varia- 
tion in one of the variables. Of 
control students, 81.2% (N = 48) 
were suspended at some time during 
the 81-82 school year, and 78% (N = 
45) reported having been suspended 
during the last school term. The 
correlation between these two mea- 
sures for the control group was only 
.04 (N = 43, n.s.). 

In short, the evidence supporting 
the validity of self-reports for 
both treatment and control groups, 
but especially for the treatment 
group, is weak. In the case of sus- 
pensions, both self-report and 
archival data converge in implying 
much lower suspensions among the 
CACYD participants, although the 
level of suspensions implied by the 
two data sources differs. 

Discussion of Interim Summative 
Results 

According to the questionnaire 
measures, participation in CACYD has 
remarkably positive outcomes. The 
results based on information from 
archives are not so positive. 

Limitations . Several limitations 
are important in assessing these 
results. The first, and most obvi- 
ous, is that the administration of 
the questionnaire by a member of the 
CACYD staff may have created a sub- 
tle and unintentional but powerful 
demand for treatment youths to pre- 
sent themselves in a positive light. 
This demand may have been absent or 
less powerful for control group 
youths . Evidence suggests that this 
is a plausible interpretation of the 
results . Treatment students' scores 
on the Invalidity Scale were signi- 
ficantly higher than control youths 
scores. This scale is intended to 
be sensitive to careless responding 
or to an attempt to fake good. The 
elevated scores do not seem to be 
due to careless responding — visual 



inspection of the questionnaire 
booklets imply that they were care- 
fully f il led out and few items are 
skipped or marked in obvious pat- 
terns. Furthermore, the differen- 
tial validity found for treatment 
and control groups for question- 
naire-based measures for which there 
exists a parallel in the archival 
data accords with a hypothesis that 
treatment students attempted to 
create a good impression or other- 
wise responded in nonveridical 
ways.^ And, the questionnaires were 
administered by a member of the Cen- 
ter staff who had frequent contact 
with the students and whom they may 
not have wanted to displease . On 
the other hand, respondents faking 
good might be expected to have 
reported fewer school punishments . 
The opposite result was found : 
Scores on the School Punishment 
Seal e are significantly higher for 
the treatment group. These, and a 
number of other interpretations, are 
possible. CACYD staff, for example 
have suggested an explanation for 
the low correlation between actual 
grades and self-reported grades that 
involves the way student perceptions 
of their performance vary over time. 
Until further work is performed, 
many such interpretations are possi- 
ble. On balance, the self-report 
results should be interpreted with 
caution. 



9. Hindelang, Hirschi, and Weis 
(1981) found differential validity 
for sel f-report measures similar to 
those examined here in their study 
of Seattle youths. Specifically, 
they found near zero validity coef- 
ficients for Black males who were 
officially delinquent. The pattern 
found here suggests limited validity 
for the control group (many of whom 
were officially delinquent) and even 
lower validity for for the treatment 
group. 
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A second limitation stems from 
the finding that treatment and con- 
trol youths differed significantly 
in the number of police contacts 
during the year prior to the study 
period. The same pattern (control 
youths being more officially delin- 
quent) holds for both the early and 
late assignment periods. This out- 
come could occur by chance given 
random assignment, or it could be a 
product of a breakdown in the ran- 
domization process. 

A third, and less threatening, 
limitation is due. to the assignment 
of different proportions of el igi- 
bles to the treatment condition at 
different time periods. Despite the 
application of a statistical control 
for period of assignment, because 
the composition of the pool of eli- 
gibles differed between the two time 
periods some residual spurious asso- 
ciation between youth characteris- 
tics and outcomes may exist in the 
results. An examination of treat- 
ment and control means separately 
for each time period suggests that 
this threat to interpretation is not 
highly plausible as an explanation 
of the results. 



10. The appl ication of a statisti- 
cal control for prior police records 
does not alter the statistical con- 
clusion: net of assignment period 
and of prior police record treatment 
youths report fewer kinds of del in- 
quent behavior than do control 
youths. Police records are not, 
however, stable covariates. The 
correlation of 80-81 police contacts 
with 81-82 contacts is low. The 
application of a statistical control 
for prior grades does not alter the 
statistical conclusion: net of 
assignment period and of prior 
grade-point average , treatment 
youths report receiving higher 
grades. . 



Implications . Although the 
interpretation of the self-report 
results is hazardous, the interim 
summative results based on the 
self-report measures suggest not 
only that CACYD participation has 
been remarkably effective in pre- 
venting del inquency , but al so sug- 
gest some explanations of its effec- 
tiveness. Several known correlates 
of delinquent behavior were affected 
by the CACYD treatment. These 
include belief in rules, attachment 
to parents and school, " self-concept 
(esteem), and involvement (measures 
of the social bond according to 
social control theory ) ; negative 
,peer influence (a measure of differ- 
ential association) ; and school 
grades. Furthermore, the greater 
amount of school rewards and punish- 
ments reported by treatment students 
suggests that supervision and the 
consequation of behavior may be a 
major way ^in^ which^ these _out comes 
can be achieved. Although site 
visit observations and interviews 
with participants are by tliemselves 
generally untrustworthy sources of 
evidence , observations and inter- 
views with students participating in 
the program nevertheless lend sup- 
port to this interpretation. Like- 
wise, observations of phone calls to 
parents when students are absent and 
of del iberate attempts of the staff 
to strengthen interpersonal ties 
with the students also support this 
interpretation. 

Discussion of Other Observations 

Our observations suggest some 
ways the project might be strength- 
ened. First , the individual ized 
education plans we examined can be 
strengthened greatly by (a) making 
them more concrete and specif ic, 
(b) reviewing and updating them more 
frequently, (c) providing specific 
consequences for meeting or fail ing 
to meet short-term obj ect ives, 
(d) ensuring that the objectives and 
consequences are clearly understood 
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by students, staff, and parents or 
guardians. 

Second, although the principles 
of effective teaching suggested by 
the project manager accord with some 
literature on effective teaching 
(e.g., Brophy & Evertston, 1974; 
Crawford & Stallings, 1978; Gage, 
1978) a formal structure for ensur- 
ing that these principles are fol- 
lowed precisely should be imple- 
mented to strengthen the academic 
component. 

Third, observations and inter- 
views with staff suggest that 
improvements in curriculum and 
teaching technology could strengthen 
the academic components. The use of 
well engineered educational technol- 
ogies such as DISTAR (Becker & Car- 
nine, 1980; Becker & Gersten, 1982) 
or various approaches to increasing 
the effectiveness of classroom 
reward structures (Slavin, 1977, 
1982, in press) could add strength 
in this area. 



4. The relationship between the 
Center and the Compton Unified 
School District was strengthened. 

5. The Center began the proce- 
dure of starting its school day with 
a youth initiated activity — a crea- 
tive and interesting innovation. 

6. Student reports imply that 
they felt positive about the alter- 
native school and that they knew 
they were expected to learn. 

7. . Warm interpersonal relations 
between students and staff were 
developed. 

8. Despite serious problems with* 
gangs in the Compton community, stu- 
dents from rival gangs generally 
co-existed peacefully in the Center. 

Recommendations 

Our observations and analyses to 
date suggest the following recommen- 
dations: 



Fourth, the intiation of a par- 
tial token economy appears to be a 
useful step. This step could be 
more fully elaborated along the 
lines developed in Achievement Place 
(Wolf et al., 1972; Levitt et al . , 
1981). 

Brief Summary of Achievements in the 
Second Year 

1. Staffing was stable during 
the second year, with no person 
leaving the Center until May of that 
year. 

2. Of students who participated 
in the Center during the 1981-82 
school year, 60% were still enrolled 
at the end of the school year. 

3. The collection of information 
by the on-site evaluator was tho- 
rough and timely. 



1. Greater use of previously 
developed educational interventions 
and curricula to increase learning 
would probably strengthen the pro- 
ject'^s academic component. 

2. The development of additional 
strategies to retain clients in the 
program for a longer period of time. 

3 . More active steps to secure 
sponsorship for the Center in the 
future must be taken if it is to 
continue to evolve. 

4. Administration of student 
que s t ionna ire s in the Spr ing by a 
person unknown to the respondents, 
especially for the treatment group. 

5. Closer replication of token 
economy systems developed and tested 
elsewhere to strengthen this project 
component, if it is to be continued. 
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6. Strengthening the 
individualized education plans by 
much more frequent review and 
greater specificity to increase 
their faithfulness to the character- 
istics of such plans described ear- 
lier. 

7. In future efforts in this 
area, making arrangements in advance 
such that demands from a sponsor for 



service provision do not conflict 
with the demands of an orderly eval- 
uation. 

8. In future efforts in this 
area, making arrangements for ran- 
domization to be conducted off-site 
within a feasible time line, and 
allowing a longer period of time to 
elapse before searching archives for 
arrests. 
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Table 1 

Overview of Tests for Differences between Treatment 
and Control Conditions 



Criterion 


Difference 






variable 


favors 


F 


df 


P 


Archival sources 










Withdrawal 


control 


0.08 


1,67 


.78 ns 


Any police contact 


control 


0.23 


1,89 


.23 ns 


Number of 


treatment 


2.69 


1,87 


.10 as 


police contacts 










Grade-point average 


control 


0.84 


1,89 


.36 ns 


beix-reppr t 








Attachment to parents 


treatment 


6.84 


1,68 


.01 ** 


Negative peer infl. 


treatment 


0.27 


1,68 


.6J) ns 


Attachment to school 


treatment 


46.81- 


1,74 


.001 ** 


Belief in rules 


treatment 


5.30 


1,70 


.024 * 


Interpersonal 


treatment 


5.60 


1,72 


.021 * 


competency 










Positive self -concept 


treatment 


2.04 


1,65 


.16 ns 


Self-reported 


treatment 


7.62 


1,54 


.008 ** 


delinquent behavior 








Self-reported 
substance use 


treatment 


5.46 


1,56 


.023 * 










belr-repor ted serious 

delinquent behavior 
School punishment 


treatment 


7.44 


1,54 


.009 ** 










treatment 


6.00 


1,73 


.017 * 


School rewards 


treatment 


6.94 


1,74 


.010 ** 


Victimization 


control 


1.83 


1,72 


.18 ns 


Invalidity 


control 


4.21 


1,71 


.044 * 


School effort 


treatment 


8.45 


1,71 


.005 ** 


Practical knowledge 


treatment 


10.55 


1,74 


.002 ** 


Alienation 


treatment 


4.88 


1,67 


.031 * 


Self-rep. grades 


treatment 


1.28 


1,74 


.26 ns 


Self -reported 


interaction^ 






reading ability 










Educ. expectation 


treatment 


1.69 


1,74 


.20 ns 


Work for pay 


treatment 


6.59 


1,74 


.012 * 


Regular job 


treatment 


1.94 


1,74 


.17 ns 


Suspension 


treatment 


24.08 


1,73 


.001 ** 


School non-attendance 


treatment 


4.31 


1,71 


.042 * 


index 










Rebel lious autonomy 


treatment 


20.81 


1,70 


.001 ** 


Involvement 


treatment 


17.31 


1,71 


.001 ** 


Note. Based on analyses 


that test 


for an 


effect of 


treat- 



ment net of period of assignment to treatment or control 
groups (see text footnote 5). The withdrawal variable may" 
not be comparable for treatment and control groups (see text 
footnote 7). 

^ Part of total self-reported delinquent behavior. 
^ Treatment group reports more school punishment. 

Nature of effect depends on period examined. 

* Statistically significant ns Not significant 

Highly statistically significant as Almost significant 
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Table 2 

An Examination of Differential Validity of Self-Report 
for Treatment and Control Groups 



Self -report 
measure 


Archival 
measure 


r 


N 


Treatment group 


Delinquent behavior 
total 


Number of police 
contacts 


-.01 


23 


School nonattendance 
index' 


Number of unex- 

cused absences, 
,^ April-end of year 


.31 


29 


Reading ability 


Plato reading 
pretest 


-.07 


29 




CACYD grades 


.05 


24 


School grades 


CACYD grades 


-.21 


25 


Control group 


Delinquent behavior 


Number of police 
contacts 


.54 


34* 


School nonattendance 


Number of absences 
CUSD, 81-82 


.23 


41 


Reading ability 


CUSD grades 


.16 


36 


School grades 


CUSD grades 


.10 


37 



*p < .01 
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Project STATUS: Second Interim Report 
D. K. Ogawa, M. S. Cook, and G. D. Gottfredson 



Abstract 



Project STATUS (Student Training Alternatives Through Urban Strate^ 
gies) is continuing its efforts at Eliot Junior High and Muir High 
School in Pasadena, California to reduce class cutting, disruptive 
behavior, dropout from school, and lack of student and parental involve- 
ment in the school. The project's five interventions: a) the Options 
class, b) the Youth Committee and Leadership Training Class, c) project 
training, d) parent involvement, and e) the Act ion/ Advisory Committee^ 
are designed to provide students with a more .meaningful educational pro- 
gram. More specific project objectives are no decrease alienation, 
increase attachment to school, self-esteem, and academic success. 

Non-equivalent control group designs are used to assess the effec- 
tiveness of the Option classes and Youth Committee at Eliot, and the 
Options classes at Muir. No evaluations are currently available for the 
other program components. Results show students in Eliot's Options 
class to be significantly lower in alienation, make higher ratings of 
their own reading ability, report more fairness and clarity of school 
rules, and have fewer withdrawals from school than students in a non-e- 
quivalent control group. It apparently also increases absenteeism for 
project students. No evidence of .effectiveness was found for Muir's 
Options class or for the Youth Committees at Eliot or Muir. We cannot 
as yet judge the effectiveness of parts of the project; numerous prob- 
lems in implementing some project components may have weakened the 
effectiveness of the planned interventions. Next year's efforts will 
include consolidating the Youth committee and Leadership Training Class 
to intensify the leadership treatment. 



Goals and Rationale 

Project STATUS (Student Training 
Alternatives Through Urban Strate- 
gies) is continuing its efforts to 
provide targeted junior and high 
school students in Pasadena, Cali- 
fornia, with a high interest curri- 
culum (Options Classes) and leader- 
ship training (Youth 
Committee/Leadership Training 
Class). Three other 



int ervent ions — rpar ent involvement , 
project training, and the action/ad- 
visory committee — are intended to 
involve parents, community members, 
and project staff in project actvi- 
ties . All of these interventions 
are designed to address five prob- 
lems found at Eliot Junior High and 
Muir High Schools: (a) class cut- 
ting; (b) disruptive behavior; 



t7on including the project's Program 

Development Evaluation plan, conver- 
sations with the project director, a 
This description of the project's report by the Project Director 

goals, rationale, and objectives (Reiss, 1982), and an earlier 

draws on several sources of informa- account of the first year's interim 
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(•) lich §f tluitnt iRVolvmtnti 

Mil Cf ) Ueh of pifMt involvMtntt 

TlicM preblcBft are thought to 
ftii4«t btciUM itutiitnts trt Mt 
iftvtlvt* In r«ltv«nt %ni MAningful 

tw%^ lofti of Mir-tittM, alUnA* 
tiM Mi Uii €OMNiUi«nt towards 
Mhool* This accords vith mpirical 
ttuditii vhicb Kayo shown corrtla*- 
tioas l^iwfOB acftdMic failuroi lov 
coMiltiioiit and attacHsitnt to school • 
and stttdont nUhthavior (8tin«- 
€hci»N» iH4; labtrbtrt \ Silbar- 
htrftt 19711 III lot k VoMi 1974; 
Mirschl, Xm). 

tn addition to oscptrioncing aca- 
demic failtttot Mny students with 
natural or traditional Itadarship 
abilitita aro soon by tho project as 
t^cMns tho appropriatt loadorship 
•bills to offoctivoly influanct 
tbolr school onvironnont. This» 
along with tho ptrcoption that tho 
ocbool nil OS aro not fair and that 
tboro aro sioohanis«t for student 
input and iavolvtviint in dtcision- 
Mkingi is boliovod to alitnatt stu-^ 
donts froa school* 

Sasod <M STATUS' thaoTy» sovoral 
objoctivos havo boon spocifiod to 
acbiovo thoir goals. Table 1 sum- 
naritos oach goal and the corres- 
ponding ob j act i vos ♦ Th is tabl e 
■abas cloar the oxtont to which the 
evaluation of this project doviates 
fron the Program Developient Evalua- 
tion CM^l» Tho objectives include 
proeoas objectivos or critical 
boncbMrka for tho implementation of 
the proJo€t» and the measures of 
goal a and ob J oc t i vo a ar e no t made 



ovaluation of this project (Carlton, 
I9S2)* Tho Carlton report provides 
a moro complete description of first 
year activities and plans* 



cleart This departure from the 
standards set for the evaluation of 
the alternative education program 
resulted in a relatively weak evalu- 
ation of Project STATUS* For exam- 
ploi the major objective of the 
Youth leadership Training Classi to 
increase students' leadership 
skillsi is lost amid a number of 
process objectives* No measure was 
apecified for thia important objec 
tive* 

Project STATUS interventions are 
baaed on the notion that intervening 
»only at the individual level or only 
at the school level cannot solve 
atudent misbehavior. The overall 
approach is designed to meet indivi- 
dual student needs as vel 1 as 
involve students, school personnel, 
parents, community members and 
organisations „ in^ affecting .school 
climate* 

Options Classes 

Sixty 7th graders and 60 9th 
graders of all academic abilities 
volunteer to participate in alterna- 
tive educational English and Social 
Studies classes* These classes uti- 
lise a curriculum focusing on cop- 
ing with authority, responsibility, 
the school, the family, and friends. 
Audio-visual presentations, field 
trips, guest speakers, role-playing, 
simulations* and action activities 
are used to enrich the curriculum 
content. This intervention, which 
attempts to provide students with 
relevant and appropriate student- 
centered materials, is primarily 
aimed at STATUS'' objectives of 
decreasing alienation and increasing 
academic achievement and self-es- 
teem. 
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Youth Committee (YC) 

Approximately 50 students identi- 
fied as traditional (e.g., student 
council members , team captains, 
etc.) and natural (e.g., gang lead- 
ers ) leaders are referred by school 
personnel to participate in the 
Youth Committee, This intervention 
is designed to provide students with 
opportunities to identify problems 
in the school and seek solutions in 
order to improve the climate of the 
school. By channeling the Youth 
Commitee members'' leadership abili- 
ties toward effective organizational 
change , project implementers theor- 
ize that these students will not 
only feel more involved with and 
committed to the school, but will 
also positively influence other stu- 
dents. 

Leadership Training Class (LTC) 

A leadership training curriculum 
was devised to provide students from 
the Youth Committee with intensive 
individual ized training in develop- 
ing decision-making , conflict man- 
agement , and communication skills. 
Approximately half of the students 
from the Youth Committee participate 
daily in this class for one semes- 
ter; then the rest of the committee 
takes the class during the second 
semester. The purpose of this class 
is to allow students to develop 
their leadership skills and apply 
them to the problems addressed in 
Youth Committee meetings. 

Staff Training 

Project staff , school administra- 
tors , and teachers participate in 
training* sessions prior to the fall 
semester to help them f acil itate 
student devel opment and involvement 
in organizational change . The 
theory behind the training is that 
school staff must be aware that 
positive student attitudes are 



essential for change and that as 
representatives of the school, their 
policies and actions are important 
in improving student attitudes and 
perceptions (Reiss, 1982) . 

In addition to these training 
sessions, project staff participate 
in on-going informal training 
throughout the year. Util izing 
weekly observation, the project 
director gives feedback to to pro- 
ject teachers, aides, and counselors 
in the areas of individual ized 
instru-ction, effective use of aides, 
and classroom participatory manage- 
ment. 

Parent Training 

Parents of students participating 
in the project were to meet on a 
regular basis to identify training 
needs and to receive training in 
maximiz ing parent input in school 
decision-making practices. The 
intent of this intervention is to 
get parents involved and to get 
parents to accept responsibility for 
what goes on in school. This inter- 
vention was not well implemented 
during the 1981-82 school year (see 
below). 

Action/Advisory Committee 

The Act ion /Advisory Committee is 
made up of parents, citizens, commu- 
nity service agencies , and other 
organizations from the Pasadena com- 
munity who identify and provide 
resources for project activities. 
In addition, members serve as role 
models and provide project students 
with vocational guidance and 
resources. Examples of activities 
include making in-class visits, pro- 
viding vocational intern experi- 
ences, and organizing on-site visits 
to various agencies. As with parent 
training , this intervention is 
designed to get the community 
involved with the school. 
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Proiect Environment and Staffing 

The project's environment, i.e., 
the implementing organization, the 
targeted school s , and the school 
district, has remained the same as 
last year (see Carlton, 1982, for a 
complete description). There have 
been, however, some changes in 
staffing . At the school sites , 
three new teachers' aides replaced 
the three aides from last year who 
left th e pr ogr am . One of th e new 
aides also left during the semester. 
Consequently, one of the high school 
teachers did not. have a teaching 
aide. 

At C?vF (Constitutional Rights 
Foundation), there was no replace- 
ment this year 'for the community 
coordinator position. One reason 
for no active recruiting was the un- 
certainty of the third year budget, 
and the inefficiency in training 
someone for a one-year position. 
However, a new administrative assis- , 
tant was hired to replace the previ- 
ous one. Since there is no on-site 
evaluator and since the school dis- 
trict's director of evaluation has 
other commitments, it is important 
for the administrative assistant to 
handle many of the evaluation activ- 
ities. 

Program Expenditures 

Figure 1 shows how STATUS appro- 
priated its $405,971 budget for the 
1981-82 school year . Salaries , 
including contractual services (con- 
sultants, stipend for training ses- 
sions , etc. ) comprise the largest 
single expediture . This 60% of the 
budget does not include the total 
sal ar ies of proj ect teachers and 
staff at the schools. A large per- 
centage of their salaries are paid 
by the Pasadena School District. 
Program materials and equipment 
account for a smal 1 por t ion of the 
budget, 7% and 2% respectively. 
Other expenditures include travel 



(2%), office operating costs (7%), 
and indirect costs (21%). 

Implementation of Proiect STATUS 

. Last year's problems in imple- 
menting the program (see Carlton, 
1982) resulted in differential 
phase s of implementation of the 
intervention components for this 
year. For example, the Options 
class is more developed than the 
parent training component . Whereas 
the Options class is at the imple- 
mentation/ fine-tuning phase , parent 
training is closer to an initiation/ 
adoption phase. Although this situ- 
ation is true at both schools. 
Tables 2 and 3 indicate that the 
degree of implementation of the 
interventions varied at the two 
schools . Table 2 is based on the 
SAES (School Action Effectiveness 
Study ) student questionnaire and 
shows that Options s.tude.nts.at Eliot 
significantly differ from non-equi- 
valent controls on measures of indi- 
vidualized instruction, field trips, 
student-teacher interactions , and 
student influence in decision-mak- 
ing. Their are no differences on 
Student Degredation (a scale measur- 
ing negative behavioral treatment of 
students). On the other hand. 
Options students at Muir differed 
significantly from their non-equiva- 
lent controls only in terms of the 
nmber of field trips. ^ This find- 
ing, however, contradicts proj ect 
r epor t s th at no f iel d tr ips were 
taken at Muir (see Table 3). The 
data in Table 3 should be inter- 
preted with caution since proj ect 
staff recorded the extent and nature 
of program activities for each stu- 
dent at the end of the school year . 
Inaccurate records and retrospective 
reporting may degrade the quality of 
the data. Other proj ect-developed 
rating systems were also examined to 
provide information as to the amount 
and quality of the Options and LTC 
classes. 
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Options Classes 

The Options classes revolve 
around a curriculum that aims to be 
of high interest to the students. 
Each unit of the Options class cur- 
riculum is based on law-related 
materials and execises, and attempts 
to deal with a significant aspect of 
the student's life, e.g., the 
family, school and the law. Each 
topic is intended to address impor- 
tant project obj ectives such as 
increasing attachment to school and 
decreasing feelings of alienation 
from school and society. ^ However, 
because curriculum units were not 
ready until the week of use in the 
classroom, teachers had little time 
to review the materials and prepare 
for class beforehand. 

Eliot Junior High . The 54 stu- 
dents and two teachers involved in 
the Options class rated each curri- 
culum unit after its presentation. 
Ten rating categories were used (see 
Table 4). These ratings were car- 
ried out using project-developed 
materials. Table 4 shows the 
results for the students' ratings. 
Two of the ratings most critical for 
the assessment of project objectives 
are ratings of curriculum relevance 
and interest. For each unit , over 
half (54-70%) of the students felt 
the material was relevant and bet- 
ween 38-70% felt the class was more 
interesting than others. We have no 
norms for these ratings, no project- 
generated standard against which to 
compare the ratings , nor are there 
ratings of curriculum made by a com- 
parison group experiencing common 
curricula. These ratings do not 
necessarily demonstrate that the 
curriculum that was implemented was 



a high interest curriculum. Is a 
38% affirmative response to "more 
interesting" good or bad? We do not 
know. As another example, about 40% 
of the students report doing work 
outside of class — without some com- 
parison group it is impossible to 
say whether the curriculum is 
inspiring more students to do more 
homework, or less. (A comparison 
group was available for the self-re- 
ported School Effort measure in the 
SAES questionnaire. No statisti- 
cally significant difference was 
found between treatment and compari- 
son students. ) Project staff 
report, however, that the results 
1, did provide feedback to the curricu- 
lum writer so modifications could be 
made for the upcoming year.^ 

Other activities of the Options 
classes included cooperation between 
Options and YC/LTC teachers in the 
planning of activities, and staff 
coordination of a mock trial compet- 
ition for the Options students which 
included non-project eighth grade 
students as jury members. 

Muir High School . Implementation 
problems at Muir began early when 
counselors mis-assigned nonproject 
students to the Options classes at 
the beginning of the semester. Pro- 
ject staff had a difficult time cor- 
rectly rescheduling the project stu- 
dents because a) it was difficult to 
re-arrange the mis-assigned stu- 
dents' schedules and b) nonproject 
students assigned to the Options 
classes enj oyed the classes and 
wanted to remain enrolled in them. 
Ultimately, about 25% of the stu- 
dents who took the Options classes 
during the first semester were ori- 
ginally designated as nonproject 



1 . For a de s cr ipt ion of the cur r i- 
culum, send requests to Elizabeth 
Salzman, CRF, 1510 Cotner Avenue, 
Los Angeles, CA. 90025 



2 • Options class teachers also 
rated the curriculum units, but res- 
ponses are based on only one or two 
teachers. 



30 



STATUS 



students, i.e., they did not meet 
the original criteria for selection 
into the program. 

Another implementation problem 
was that one teacher did not have a 
teaching aide. In addition, the 
project director reports that alter- 
native education techniques have not 
been completely adopted at Muir, and 
teachers there do not spend as much 
time planning together as teachers 
at Eliot. 

Table 5 shows the results of the 
students' ratings, of each unit .of 
the Options curriculum as taught at 
Muir. Depending on the unit, bet- 
ween 65-79% of the students agreed 
that the information presented was 
relevant. Of the students respond- 
ing, 77-89% felt the class was more 
interesting than others. Project 
staff reported that the ratings 
proved useful in modifying the cur- 
riculum. 

Youth Committee/Leadership Training 
Class 

The YC/LTC was only weakly imple- 
mented in 1981-82. There are sev- 
eral reasons why. As was the case 
during the first project year there 
were problems in scheduling YC meet- 
ings. Meetings could not be sche- 
duled after school because this con- 
flicted with other activities. 
Lunch period was too short. Teach- 
ers objected to having students 
attend YC meetings during class 
time— despite these students being 
identif ied as leaders , many lack 
academic skills and cannot afford to 
miss class. However, for lack of a 
better time , meetings at both 
schools were held during class time. 
Consequently , YC meetings were hel d 
infrequently . A second problem was 
that few of the YC students got 
involved in project activities: out 
of 50 YC members at Eliot, only 4 
students participated in presenta- 



tions and 7 participated in school 
committees. At Muir, 17 out of 77 
members participated in presenta- 
tions and 15 participated in school 
committees. This suggests that stu- 
dents may have been attending the 
meetings only to get out of class. 

In addition to problems in sche- 
duling YC meetings, not all YC mem- 
bers could be scheduled into the 
LTC, an elective course . This 
course was only offered one period 
per day and often conflicted with 
other courses. Some of the YC mem- 
bers who did take the LTC subse- 
quently dropped out of the YC. The 
t, scheduling and drop-out problem 
resulting in only a few students 
receiving training and experience in 
the use of leadership skills, per- 
haps subverting the YC/LTC as a 
school change intervention. Accord- 
ing to the project's theoretical 
model, . systematic ^changes., ^cannot 
occur within an organization with 
only a small number of trained indi- 
viduals. 

Another problem in implementing 
the YC/LTC was that the two project 
paraprof essional counselors , who 
coordinated the YC/LTC activities, 
may have taken on too many responsi- 
bilities for their half-time non- 
professional status . 

Besides overseeing the YC/LTC, 
the paraprof essional counselors also 
served as school counselors for the 
Options and YC/LTC students. The 
program design called for the para- 
professional coumselors to arrange 
parent conferences and make home 
visits if students were failing aca- 
demically, cuttitag class, or behav- 
ing badly, but almost no conferences 
or home visits took place. Records 
show that at Eliot only one home 
vist was made all year, and at Muir, 
two. At Eliot, 25 students received 
on-going counsel ing from the para- 
prof essionals. At Muir , the para- 
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professional counselor contacted all 
of the Options students at least 
once at the beginning of the semes- 
ter. During the course of the year, 
15 students at Muir were called, in 
once for behavioral problems j 
another two students were called in 
twice. 

A further problem with the imple- 
mentation of the YC/LTC is that the 
counselors had little exposure to 
the leadership training curriculum. 

To summarize the efforts of the 
pa:, aprof essional counselors, they 
only worked half-time , were unfamil- 
iar with the leadership training 
curriculum, provided little or no 
documented counseling, and made a 
total of only three home visits. It 
is likely that unless they engaged 
in other, undocumented activities, 
they had little impact on project 
students. 

Eliot Junior High School . A 
total of 50 students participated in 
the YC during the year and addressed 
such issues as security and safety 
at school, longer lunch periods, and 
more elective courses for 8th grad- 
ers. Twenty-eight of the YC members 
took the LTC. Scheduling difficul- 
ties led to a late start-up date for 
the class. As a result, first sem- 
ester LTC students remained in the 
class for the entire year, with no 
new students added the second semes- 
ter. 

Table 6 shows the results for the 
student ratings of the leadership 
training curriculum. Ratings of 
relevance , interest , and leadership 
training received are of particular 
interest to project staff. Overall, 
83% of those responding felt the 
information presented was relevant, 
77% found the class more' interesting 
than others, and 91% felt they had 
developed leadership skills. Unfor- 
tunately, we do not know whether the 
students did in fact improve their 



leadership skills. 

One result in the ratings that 
may be important for program devel- 
opment efforts is that the ratings 
at El io t or general ly lower than 
those at Muir: nineteen of twenty- 
five ratings favor Muir. Even this 
comparison should be interpreted 
very cautiously because of the small 
number of students reponding to the 
survey. Percentage agreement scores 
are unstable when the n'^s are as low 
as they are here. 

Muir High School . The 77 members 
of the YC addressed issues such as 
class and school cutting and student 
"pride. The LTC was held daily for 
16 YC members each semester. Eight- 
y-two percent of the second semester 
LTC students rated the information 
presented in class as relevant; 85% 
found the class more interesting 
than other classes at Muir. All of 
the students^ responding reported 
that they had developed the ability 
to lead a group (see Table 6). 
Again, without a control group, 
there is no clearcut way to inter- 
pret these ratings. 

Staff Training 

Dur ing the school year , several 
training sessions^ were held with 
project/ staff, teachers, and school 
administrators. A total of 17 for- 
mal training meetings with proj ect 
teachers, counselors, and aides were 
held. Over 40 individual meetings 
were held with project teachers and 
the project director to discuss the 
in-class observations made weekly by 
the project director. 

Parent Training 

Two meetings at each school for 
the parents of STATUS students were 
held this year. Approximately 50 
parents were involved in these meet- 
ings which were run by the project 
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director and principals of the pro- 
ject schools. An overview of the 
program was given and an description 
of how parents could help the pro- 
ject and how the project could help 
parents. The program design called 
for the holding of parent training 
workshops, but none were held. No 
parent committees were established. 

Action/Advisory Committ ee 

This year the action and advisory 
committees were combined into one 
committee. Meetings were held every 
six to eight weeks with an average 
attendance of 15. The committee 
revised and distributed the commu- 
nity resource guide. The committee 
also held a public relations meeting 
at the superintendences house to 
expand the project's base in the 
community. 

Summary 

For the Options and YC/LTC inter- 
ventions, obstacles internal and 
external to the project influenced 
implementation. A major external 
force was schedul ing : proj ect stu- 
dents could not be scheduled into 
project classes, non-project stu- 
dents were scheduled into project 
classes , and convenient meeting 
times for the Youth Committe could 
not be found. An internal obstacle 
to faithful project implementation 
was the reluctance of project staff 
to adopt teaching and counseling 
styles advocated by the project 
director . In addition, project 
implementers were inexperienced with 
the curriculum developed for the 
Options classes and the Leadership 
Training Classes. Thus, project 
staff did not adopt project educa- 
tional techniques, and were not 
experienced in project curricula. 
It appears, therefore, that these 
components were not implemented as 
designed. 

A further implementation 



difficulty was the ignorance of pro- 
ject St af f about the goal s of the 
project. It is only with the start 
of the third project year that other 
project staff besides the proj ect 
director have begun to understand 
the goals and objectives of the pro- 
gram. Final ly , a lack of direct 
project control over selection and 
supervision of in-school staff, and 
a shortage of personnel to coordi- 
nate all aspects of the interven- 
tions al so contributed to poor pro- 
ject implementation. 

Interim Outcomes 

Int er im ev al ua t i on re sul t s hav e 
two potential uses : they can pro- 
vide information as to the effec- 
tiveness of a project to date, and 
may serve as mater ial for fur ther 
program devel opment . The extent to 
which the information is useful 
depends in large part on the ade- 
quacy of the evaluation design. For 
the CRF project, strong evaluations 
were agreed to, but were considera- 
bly weakened by non-random assign- 
ment to treatment and control, and 
multiple treatment for project stu- 
dents. 

Evaluation Design 

Options Classes . Potential 
Options participants were recruited 
by project staff prior to the fall 
semester. Interested students were 
required to submit an appl icat ion 
and acquire parental permission to 
participate in STATUS. From this 
pool of appl icants project staff 
selected students who were represen- 
tative of the school population, 
i.e. , participants included students 
of all academic abilities, races and 
behavioral dispositions. The pro- 
ject agreed to randomly assign stu- 
dents to treatment and control from 
this referral pool . Randomization 
was not adhered to at either Eliot 
or Muir. 
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At Eliot, students were randomly 
assigned to treatment and control 
from the pool of screened referrals. 
Roster updates collected in October 
indicated little relationship bet- 
ween the original random assignment 
and the students designated as 
"treatment" and "control" by project 
staff. The reason for this is 
unclear. What is clear is that the 
evaluation of Eliot's Options 
classes involves a comparison bet- 
ween treatment students and a non-e- 
quivalent control group accompanied 
by an unmeasured selection bias. 

The experimental evaluation of 
the Options classes at Muir broke 
down when not enough students 
received parental permission to par- 
ticipate. The non-equivalent con- 
trol group at Muir therefore largely 
consists of students whose parents 
did not wish them to receive alter- 
native education. The differences 
between such students and those 
whose parents did grant permission 
cannot be known with any certainty. 

The YC . As with the Options 
classes, YC participation is on a 
voluntary basis and is open to all 
students. Faculty, administrators, 
parents, and students referred stu- 
dents to the YC. The criteria for 
referral was that the student was 
identified as a natural or potential 
leader. LTC students were non-ran- 
domly selected from among those stu- 
dents recognized as the most promis- 
ing leaders in the YC. Selected 
students also had to be able to fit 
the LTC class into their school 
schedules. Thus the LTC students 
are a select group, and no convinc- 
ing control group for them could be 
developed — LTC students will not be 
evaluated seperately from YC stu- 
dents at large. 



We do not know how comparison 
group students were selected for the 
Eliot and Muir Youth Committees: 
students from the referral pool and 
consent pool had originally been 
randomly assigned, but different 
students were ultimately designated 
by the project as treatment and con- 
trols. Students assigned to control 
became treatment, and new students 
appeared in both treatment and con- 
trol groups . To compound these 
problems, at Muir only 6 controls 
were identified . No sensible com- 
parison is possible between approxi- 
mately 50 treatment students and 6 
controls. Therefore the Muir YC 
cannot be evaluated. 

Summary of Evaluation Designs . 
For the Options classes, quasi-ex- 
perimental non-equivalent control 
group designs describe the evalua- 
tions at both Eliot and Muir. The 
evaluation of_ the YC_ at Eliot can 
also be characterized as a non-equi- 
valent control group design. No 
evalution design was implemented for 
the Muir YC, or for either school's 
LTC. 

Measures . Measures come from 
several sources: a) the SAES ques- 
tionnaire, b) school records, c) 
project questionnaires, and d) pro- 
ject records. The maj ority of mea- 
sures are scales and individual 
items from the SAES student questi- 
onnaire. Test scores, attendance 
and grades were taken from school 
records. Project questionnaires 
include student and teacher evalua- 
tions of the curriculum, student 
development of leadership competen- 
cies, and a teacher observation 
checklist. Information from project 
records will be used to determine 
the number of parent meetings held, 
and the number of policy changes 
initiated by parents and students. 
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Results 

Equivalence of the Control 
Groups * Tables 7 and 8 give pretest 
comparisons between the various 
treatments and their available con- 
trol s . The treatment and non-equi- 
valent control groups appear fairly 
similar on the measures taken (these 
pretest measures were scored in a 
different — but equivalent — metric 
than similarly-named outcome mea- 
sures reported later). The Eliot 
Options students do not differ from 
their controls on any of the 12 psy- 
cho-social measures, and the Youth 
Committee students differ from their 
controls only on a measure of 
Attachment to Parents : YC students 
are more attached to their parents 
than the non-equivalent controls are 
to theirs. At Muir, the Options 
students differ from their controls 
on measures of Practical Knowledge 
and Rebellious Autonomy: The treat- 
ment students report more practical 
knowledge, and are less rebellious. 
Both of these comparisons, there- 
fore, favor the treatment group. 

The evaluation designs for the 
Eliot Options classes and Youth Com- 
mittees, and the Muir Youth Commit- 
tees are non-equivalent control 
designs. They are not true experi- 
ments. Besides the measured differ- 
ences that do exist, other, unmea- 
sured, differences may well exist. 

Project Goals . Tables 9 through 
11 show treatment and non-equivalent 
control group comparisons on several 
measures of STATUS' goals for both 
Eliot and Muir. Three measures were 
used to assess class cutting (see 
Table 9). Options students at Eliot 
had signif icantly more unexcused , 
i.e. , non-illness and non-medical , 
absences than non-equivalent control 
students. Statistically, the prob- 1 
ability of this difference occurring 
by chance is less than 5 times out 
of 100 (p< .05). Although not sta- 



'tistically significant, these 
students also reported themselves as 
cutting classes and school more than 
did the controls. Options students 
at Muir do not signif iciantly differ 
from their non-equivalent controls 
on any attendance measure . No sig- 
nificant effects on attendance were 
found for the YC at Eliot. 

No signif icant differences were 
found in the amount of disruptive or 
delinquent behavior between Options 
and non-equivalent conf rol students 
at Muir and Ell iot , nor between YC 
students and non-equivalent controls 
at Eliot (see Table 10). 

There was a significant differ- 
ence between Options and non-equiva- 
lent control students at Eliot in 
terms of the number who withdrew 
from school (see Table 11). Signi- 
ficantly fewer Options students 
withdrew from school (p<.01). Par- 
ticipants in the YC at Eliot and the 
Options classes at Muir did not dif- 
fer from their non-equivalent con- 
trols in the number of withdrawals. 

With no design, we do not know 
how Muir^'s YC affected the project^'s 
goals. 

Unlike last year, the Youth Com- 
mittees at Eliot and Muir were able 
to get students involved in school 
decision-making. This year, there 
were three student-initiated pol icy 
and procedural changes put into 
practice . However , the pro j ect was 
unable to increase parental involve- 
ment in school decision-making prac- 
tices at either school — no parent 
initiated policy or procedural 
changes were put into practice. 

Pro 1 ect Objectives . The results 
of treatment and non-equivalent con- 
trol group comparisions on STATUS'' 
obj ectives as measured by the SAES 
student questionnaire are shown in 
Tables 12 and 13. Options students 
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at Eliot are significantly less 
alienated (p<.05) than controls, and 
are more likely to report that 
school rules are fair (p<.01) and 
clear (p<.01; see Table 12). There 
are no differences on scales measur- 
ing Self-esteem or Attachment to 
School • Muir Options students do 
not differ from their controls on 
any of the above measures. 

The YC at Eliot produced no mea- 
surable effects on measures of 
Alienation, Self-Esteem, Attachment 
to School, or the Clarity or Fair- 
ness of School Rul es . We cannot 
assess the effectiveness of the YC 
at Muir at meeting the program's 
psycho-social objectives . 

In terms of academic achievement. 
Options students at Eliot reported 
themselves to be higher in reading 
abil ity than did the non-equivalent 
controls (p<.01; see Table 13); they 
did not report, however, getting 
higher grades, or receive them. No 
measurable effects on school 
achievement were found for the 
Options program at Muir, or the YC 
at Eliot. We cannot judge the 
impact of Muir's YC on academic 
achievement. 

Outcomes of Theoretical Interest . 
Table 14 shows the results of treat- 
ment and non-equivalent control 
group comparisons on outcomes not 
directly addressed by project STATDS 
but which are of theoretical impor- 
tance to the prevention of del in- 
quency. These outcomes also provide 
a check on any unanticipated nega- 
tive or positive effects of the pro- 
ject. 

Options class students at Eliot 
differ from the non-equivalent con- 
trol s only on a measure of Interper- 
sonal Competency : Options class 
students report themselves to be 
more interpersonal ly competent than 
do the controls. They do not differ 
on scales measuring Attachment to 



Parents, Parents' Emphasis on Educa- 
tion , Bel ief in Conventional Rul es , 
Life Competency , Internal Control , 
Negative Peer Influence , School 
Effort , Involvement in Conventional 
Act iv i ties , Rebel 1 ious Autonomy , 
Victimization, School Punishments , 
School Rewards, or Invalidy (a mea- 
sure of truthfullness in reporting). 

Eliot YC members report them- 
selves to be more Involved in Con- 
ventional Activities than do their 
controls. This is probably an 
implementation measure, and suggests 
that the Eliot YC members are becom- 
ing involved in a wider range of 
„ exer ience s . The YC students al so 
report signficantly lower attachment 
to their parents . Why th is shoul d 
be the case is unknown. Although it 
is conceivable that the program is 
causing the students not to care for 
their parents , the more likely 
explanation, is^ that_this is a ..pre- 
existing difference arising from the 
non-equivalent nature of the design. 
None of the other measures on Table 
14 show any differences between 
Eliot YC treatment and controls. 

The Muir Options class does not 
differ from its designated non-equi- 
valent control on any of the four- 
teen psycho-social outcomes found in 
Table 14. 

Summary . The interim outcome 
evaluation may be summarized as fol- 
lows: 

1. Comparisons between Eliot 
Options class treatment students and 
non-equivalent controls suggest that 
the class results in fewer withdra- 
wal s from school , decreased al iena- 
tion, greater interpersonal compe- 
tency, higher perceived fairness and 
cl ar ity of school rul es , but more 
unexcused absences. This interven- 
tion may be also be increasing stu- 
dents'' perceptions of their own 
reading abil ity of students , but it 
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is not affecting grades, or disrup- 
tive behavior. 

2. The evaluation of the Muir 
Options class provides no evidence 
that it has any effects on its par- 
ticipants — no differences were found 
between the treatment and non-equi- 
valent controls on any of twenty- 
seven outcome measures. 

3. Comparisons between Eliot YC 
students and their non-equivalent 
controls show that YC students 
report more Involvement in Conven- 
tional Activities (a program imple- 
mentation measure ) , but less attach- 
ment to their parents. The latter 
effect is probably a pre-existing, 
between group difference. The 
interim evaluation found no evidence 
that this intervention is affecting 
attendance, academic achievement, 
disruptive behavior, or psycho-so- 
cial adjustment for participating 
students. 

4. No convincing evaluation is 
available for the Muir YC, or the 
parent involvement, project train- 
ing, and action/advisory committee 
components of the project. 

5 . Implementation data impl ies 
that the Eliot Options class was 
more strongly implemented than the 
Muir Options class. Nevertheless, 
student ratings suggest that the 
Muir students enjoyed the class 
more. 

Discussion 

Caveats to the Evaluation Results 

As discussed earl ier , there were 
significant breakdowns in the origi- 
nal experimental designs for the 
Eliot Options classes and YC, and 
the Muir Options classes. Despite 
their similarity on the pretests 
avail able , unmeasured differences, 
especially motivational differences , 
may exist between the treatment and 



resultant non-equivalent control 
groups. All results (and non-re- 
sults) must be interpreted in this 
light. 

What Accounts for the Results ? 

Four factors may account for the 
lack of effectiveness of the Muir 
Options class as compared to the 
more positive effects found for the 
Eliot Options class. First , it may 
be that the curriculum is more 
effective on younger students, i.e., 
7th graders as opposed to 9th grad- 
ers. 

Second, an undetermined number of 
control group students received pro- 
j ect services at Muir. This is a 
potential explanation of a lack of 
differences between the control and 
treatment students on the outcome 
measures. To examine this possibil- 
ity , analyses were conducted using 
only non-equivalent controls who 
received no project services. This 
liberal analysis still uncovered no 
differences between the treatment 
and "no-treatment" controls at Muir. 

A third reason why Muir's Options 
class appears to be less successful 
than Eliot^s , is that Eliot^s class 
may have been somewhat better imple- 
mented (see Tables 2 and 3). 

A fourth possibility, of course, 
is that the few significant findings 
(both positive and negative) for the 
Eliot Options class are due to pre- 
existing differences between treat- 
ment and non-equivalent control stu- 
dents. Because we do know how 
treatment and controls were selected 
we cannot conclusively attribute 
program effects (or the lack of 
them) to the program. 

The null findings for the YC on 
individual level measures at Eliot 
may be misleading because the empha- 
sis of this intervention is on 
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school-level changes, unlike the 
Options class, which focuses on 
individual -level changes. It is 
possible, although unlikely, that in 
a school climate sense, the YC may 
affect the control students as much 
as the treatment students • 

The puzzling negative effect that 
the Eliot Options class appears to 
be having on attendance may be the 
result of the positive effect the 
program is probably having on with- 
drawals from school . The program 
may be successfully retaining stu- 
dents in the program that would have 
dropped out ; these students may 
still skip school more than students 
that would have remained in school 
without the program's support. In 
other words , the "negative" effects 
on absences may be a paradoxical 
result of keeping Options students 
enrolled in school. 

Implications of the Results 

Plausibility of STATUS' Theory . 
Although we cannot unequivocal ly 
attribute the positive effects found 
for the Eliot Options class to the 
class, the program does appear to 
have met certain project objec- 
tives — decreasing alienation, and 
increasing the use of individualized 
instruction. It also appears to 
increase students' perceptions of 
their reading ability. It also 
probably is reaching a project goal 
of decreasing withdrawals from 
school . Al so , even the El iot 
Options class has had no measurable 
effect on delinquency. In short, we 
do not know as yet whether the 
theory behind STATUS as a del in- 
quency prevention project will ulti- 
mately prove useful. 

The theory underlying the YC is 
untested in results to date. No 
evidence is available ' to suggest 
that this portion of the project is 
effective; it was not implemented in 
strong form. 
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Program refinements for 1982-83 . 
Because the results suggest that the 
YC and LTC interventions as imple- 
mented in 1981-82 did not affect 
students , program changes have been 
made for the 1982-83 year. The YC 
and LTC will be combined next year 
into a single intervention, which 
will serve only 30 students. Fewer 
students will be served, but the 
intervention will be intensified by 
having a small number of students 
meet everyday to receive the leader- 
ship training and practice in appli- 
cation. We are hopeful that this 
consolidation may eliminate the pro- 
gram implementation and evaluation 
I, design breakdowns encountered this 
year. 

Modifications were made for both 
the Options and LTC curriculums, 
based on the feedback from the stu- 
dent and teacher ratings collected 
by the project. A greater emphasis 
will be made on addressing STATUS' 
objectives in each curriculum unit. 

Evaluation Strategies for 1982-83 

The structure of the PDE plan has 
been modified this year to make it 
more useful to the proj ect staff. 
In addition, clearer specification 
of the proj ect 's action theory has 
resulted in more coherent objectives 
and helped guide intervention stra- 
tegies. 

As with last year, a quasi-exper- 
imental design will be implemented 
next year to assess the effective- 
ness of the Options and Youth Lead- 
ership Training class components. 
We were largely unsuccessful at 
changing the procedures for place- 
ment of students into treatment and 
control conditions for the 1982-83 
school year, but an early assessment 
of the equivalence of the groups 
shows no maj or differences between 
treatment and control students on 
the few outcomes we analyzed. 
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Problems Project STATUS is Encoun- 
tering 

As discussed earlier, Project 
STATUS has experienced major prob- 
lems in implementation. A second 
major problem concerned evaluation 
issues. One issue is that the pro- 
ject director feels there is a 
trade-off between implementation 
activities and evaluation activi- 
ties, status's small staff makes 
this problem particularly acute. 
Another issue is the District evalu- 
ator 's attitude toward the national 
evaluation. He feels the evaluation 
effort is larger than the program 
itself and has strongly questioned 
the validity of the evaluation and 
the use of the SAES questionnaire. 
This resistance may have diverted 
energies from program implementation 
to evaluation issues. A final eval- 
uation issue is providing data to 
the national evaluators in a timely 
fashion. Besides a lack of manpower 
to collect the data, lags in updat- 
ing school records creates a data 
management problem. Plans have been 
made for the upcoming year to 
address these obstacles. 

Successes Project STATUS is Encoun - 
tering 

The treatment and non-equivalent 
control comparisons indicate that 
the Options class at Eliot is proba- 
bly successfully achieving some of 
its objectives. School administra- 
tors, teachers, and parents have 
reported marked improvement in 
Options classes participants; they 
feel that these students have newly 
acquired academic leadership and 



interpersonal skills (although, as 
yet , we have been unable to verify 
such changes with hard data). Stu- 
dents have given presentations at 
board meetings and have competed in 
a. Mock Trial Competition. These 
successes have helped turn initial 
skepticism of the program by school 
and district staffs to support. As 
a consequence, STATUS has been writ- 
ten into the Muir School Improvement 
Plan, funded by the state to improve 
the school environment . In addi- 
tion, a description of* the project 
has also been included in "A Compen- 
dium of Character Educatioti Programs 
in California Public Schools." 

Institutionalization 

Even before it was known whether 
the project would be funded by OJJDP 
for the 1982-83 school year, the 
Options classes at both Eliot and 
Muir were .included^ jLn_ the .master 
schedule for the upcoming year. The 
English Department at Muir is con- 
sidering the adoption of project 
materials to be used in other 
classes. Other schools in the dis- 
trict have also expressed interest 
in STATUS. 

Although next year will be the 
last year of federal funding for 
Project STATUS, there are seme prel- 
iminary indications that project 
STATUS or some aspects of the pro- 
ject may be adopted by the project 
schools ^nd other schools as well. 
The Options class shows potential. 
Careful implementation and monitor- 
ing of other aspects of the program 
will be necessary to realize the 
program's optimal strength. 
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Figure 1 

Project STATUS* Budgeted Expenditures for 1981-1982 




Table 1 



Project STATUS' Cools, Objectives, and Measures 



Goal/Objcctive 



Measure 



I. Cunl: Dccrea«;e class cutting* 



I. A. SAES sLudonL questionnaire 
0. Attendance data 



Objectives : 



1. Increase use of individualized instruction plans. 



l.a. Count of individualized educational plans for 
student 

b. SAES student questionnaires 



2. Increase use of appropriate student centered materi- 
als. 



2. a. Teacher evaluation of CRF material 
b. Student evaluation of CRF material 



3. Increase use of high interest curriculum. 



3. a. Teacher evaluation of CRF material 
b. Student evaluation of CRF material 



A. Increase self-esteem. 



4. a. SAES Student questionnaire 



5. Increase academic achievement. 



5. a. CPA 

b. CTDS scores 

c. SAES student questionnaire 



6. Decrease alienation. 



6. a. SAES student questionnaire 



7. Increase attachment to school 



7. a. SAES student questionnaire 



II. Goal: Decrease disruptive student behavior. 



II. A. SAES student questionnaire 



Obj ect ives i 



1. See objectives 1-7 above. 
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Table 1 (cont.) 



Gonl /Objective 



Mensurc 



III. Goal: Increase student involvement in decision making III. A. Number of student initiated policy and procedure 
levels. changes put into practice 

Objectives : 



I. Decrease student alienation. 



l.a. SAES student questionnaire 



2. Increase classroom participatory management. 



2. a. Student self-report 

b. Observations of classroom procedures by project 
director 



3. Increase leadership competencies of youth committee 
members . 



3. a. Student self-report 



4. Increase perceptions of fair. and clear school rules. 



4. a. SAES student questionnaire 



IV. Goal: Decrease drop-out rate. 



IV. A. Attendance data 



Objectives : 



1. See Goal IIi objectives J-7. 



2. Increase parents' awareness of their child's capa- 
bilities . 



2. a. Number of times parents participate in meetings 
with project teachers and counselors 
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Table 1 (cont.) 



Goal/Objective 



Measure 



V. Goal: Increase parental involvement. 



V.A. Number of part.'nL itiiliated policy ctiangos piiL inLci 
practice- 

B. Number of parent initiated procedure changes put inLo 
practice 



Objectives : 



1. Increase number of opportunities for parents to par- 
ticipate. 



l.a. Number of parent meetings held 

b. Number of parenL committees 

c. Number of pnrenLs alt ending moe Lings 



2. Increase parents' skills to participate meaning- 
fully. 



3. Increase community involvement. 



2. a. Number of PET meetings held 

b. Number of parents attending training sessions 

/ .. 

3. a. Number of Advisory/Action committees held 
b. Number attending mccLings 



Note. Goals and objectives were extracted from the Program Development worksheeLs and Crom, Pr oject STATUS : An over view, 
paper presented by David Reiss at the annual meeting of the American Educational Research Association in Now York, 1982. 
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Table 2 



TnMiiimU m\ Cimirnl CunipnrjMonf; on 
Scvi'f.'i! I'.AI'.S ihnvil IfMiii.iirr nl' h'ii|i'Cl Sl.iLit:;' liiinli'iiii»nlalliin ObjectlvcK 



Tri'alinciit 
<ind Group 


Imllviclunlucd 
Instruction' 
H Sd N 


Field Trips'' 
M Sd 


N 


Stiulont-Tpnc|i 
Intcr.icliiiii 
M Sd 


\cr 

N 


Student ^ 
Degredation 
K Sd N 


Student 1^ 
Influence 
M Sd N 


liLIOT JUNIOR lliai SCIIOOI, 


Options 
































Trcatmi'iit 


2.68* 


1,90 


59 


1.59« 


1.04 


59 


1.52** 


1.37 


60 


2.18 


1.57 


57 


2.71* 


1.39 


56 


Conlrol 


1.84 


1.86 


51 


.49 


.81 


55 


.90 


.99 


51 


1.86 


1.90 


51 


1.83 


2.04 


48 


YC i l.TC 
































Truatmcnt 


1.89 


1.86 


71 


.94** 


.96 


50 


1.48 


1.05 


48 


2.80 


1.75 


45 


2.36 


2,00 


45 


Control 


1.61 


1.59 


/i9 


.47 


.73 


74 


1.16 


1.15 


79 


2.47 


1.69 


72 


2.21 


1.81 


71 


■ mini men sciini)!, , 


Options 








a 
























Treatment 


2.33 


1.93 




1,61 


.98 


54 


1.60 


1.26 


52 


'2.10 


1.60 


51 


3.02 


1.97 


51 ■ 


Control 


1.67 


1.75 


54 


.42 


.66 


55 


1.17 


1.19 


53 


1.57 


1.45 


54 


2.34 


2.22 


50 



Note ; Due to small number of non-equivalent controls (n=6) meaningful comparisons cannot be made between Youth 
Coniinktcc Hiembcrs and controls at Huir High School 

*^ndividual questionnaire items 

individual level student scales 

**p<.01 
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Table 3 



Average Number of Times Project Status 
Students Participated in Progiai Activities 



Seiesters in Options Semesters in LTC YC Meetings Task Forces Field Trips 

Group M.- SD N M SD N M SD N M SD N M SD N 



ELIOT JUNIOR HIGH SCHOOL , 

Options .91 .94 114 .09 .41 114 .45 1.92 114 .11 .49 114 . .08 .36 114 

YC/LTC .10 .35 140 .36 .74 140 2.70 3.97 140 .64 1.07 140 .43 .70 140 



MUIR HIGH SCHOOL 

H Options .91 .95 108 0 0 108 .39 1.29 108 .01 .10 108 0 0 108 

w YC/LTC .03 .23 91 .25 .44 91 2.84 2.35 91 .07 .29 91 0 0 91 



Note . The information presented here is based on end of the year project reports on the nature and extent of progran 
activities for each student. This includes both treatment and control groups since some control students participated in 
other aspects of the program. 
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Table 4 



Options Students 
Ratings of High Interest Curriculum 
at Eliot Junior High 



Item 



Unit I - Family 
Percent 

Agree N_ 



Unit II - Law 
Percent 

Agree N 



Unit III - Person 
Percent 

Agree N 



Unit IV - School 
Percent 

Agree N 



Understood purpose 94Z 

Information was relevant 70 

Eiijoyed activities 82 

Did outside work 39 
Read and understood materials ?4 

Speakers, fiks helpful 92 
Would recommend class 

to friends 59 
Class more interesting 

than others 54 
Materials different than 

other classes 90 
Class lived up to expecta- 
tions 65 



127 
125 
129 
126 
123 
114 

127 

126 

123 

124 



54 
88 
43 
95 
88 ■ 

55 

70 

93 

85 



42 
42 
41 
42 
41 
42 

43 

43 

43 

40 



66 
93 
37 
93 
93 

71 

39 

90 

66 



42 
38 
40 
41 
42 
42 

41 

41 

41 

35 



61 
93 
39 
100 



66 
63 
90 
73 



Note- Rating forms were given to both English and social studies Options students at the end of each unit. 



/ 



Sii 



40 
41 
42 
41 
40 
41 

41 

41 

40 

37 
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Table 5 



Options Students' Ratings of 
High Interest Curriculum at Muir High School 





unic 11 


~ Law 


i. TTT 

Unit III 


- School 


Unit IV - 


Job 


Unit V - 


Life 




Percent 




Percent 




Percent 




Percent 




item 


Agree 


N 


Agree 


N 


Agree 


N 


Agree 


N 


uuuciouuuu uurpubc^ 


Wfy 


IIZ 


93^ 


102 


97% 


33 


lOOZ 


18 


iiauiiiidLiun Wdo relevant 


Oj 


IIU 


66 


101 


79 


33 


65 


17 


tnjoyea activities 


91 


111 

111 


95 


102 


88 


32 


94 


18 


JJIU UULSiQc WOItC 


01 

LI 


1 (\1 

10/ 


23 


104 


52 


33 


56 


16 


ixCau duu uIIUcidLUUQ 












materials 


96 


no 


96 


103 


94 


33 


94 


18 


Speakers, films helpful 


96 


110 


07 




Q/i 


JJ 


o9 


1 0 
18 


Would recommend class to 












friends 


93 


110 


90 


104 


88 • 


33 


94 


18 


Class more interesting 


















than others 


77 


110 


87 


104 


78 


32 


89 


18 


Materials different from 














other classes 


88 


109 


88 


104 


88 


32 


100 


18 


Class lived up to expecta- 


















tions 


79 


105 


7^ 


98 


84 


31 


89 


18 



Note. Rating forms were given to both English and social studies Options students at the end of each unit. 
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Table 6 



Student Ratings of LTC Curriculum 
at Muir High and Eliot Junior High 



Muir High^ Eliot Junior HigH" 



Item Percent 

Agree 


IN 


P OT" r» o n t* 
A Ofc^e^ 


N 

Vi 


Understood purpose 


100% 




96% 


23 


Information was relevant / 


82 


1 1 
J. J. 


83 


23 


Enjoyed activities 


100 


1 '\ 

1 nj 


91 


23 


Did outside work 


85 


1 0 


52 


23 


Read & Understood materials 


92 




87 


23 


Speakers, films useful 


92 


1 9 


86 


21 


Would recommend this class to friends 


100 


1 

1 0 


91 


22 


Class more interesting than others 


. 85 


1 J 


77 


22 


Materials different from other classes 92 


1 9 


q 1 




Involved in rule development 


83 


1 o 

1 2 


7 1 


91 


Involved in class constitution dev*t. 


33 


1 0 


H O 


9 1 


Involved in planning 


30 


1 n 


1 U 


9 1 


Involved in making suggestions 


7b 


1 9 


82 


22 


Class lived up to expectations 


100 


13 


45 


20 


Involved in selecting school issues 


73 


11 


75 


21 


Expressed opinions 


100 


13 


67 


21 


Class conducted by students 


85 


13 - 


. 21 _ _ 


_21 _ _ 


Different points of view heard 


100 


13 


77 


22 


Decisions based on general aggreement 


62 


13 


82 


22 


Students responsible for ovm behavior 


83 


12 


52 


21 


Students respect different opinions 


92 


13 


50 


20 


Little wasted time 


85 


13 


45 


22 


Students decide plan 


100 


13 


73 


22 


Equal group representation 


69 


13 


55 


22 


Developed ability to lead group 


100 


13 


91 


22 



Note: Leadership training class students were given course evaluations at the 
end of the year. 

^Responses are from second semester students only. 
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Table 1 

Means and Standard Deviations 
on Fall '81 Proteat Mcaaures- 
Ellot 



Treatment 



Options Class Youth Comittee 





Treatment 






Control 




Treatment 






Control 






M 


SD 


N 


M 


SD 


N 


M 


SD 


N 


M 


SD 


N 


AlUflAClon 


.92 


1.08 


44 


.84 




.83 


35 


1.23'" 


1.18 


48 


1.14 


.94 




71 


Attflchmcnt to Parents 


1.28 


2.49 


42 


1.94 


3.06 


37 




2.80 


48 


-.71 


4.07 


71 


Negative Belief (Non<Bollef In 

Rill Afl^ 


1.41 


1.37 


42 


1.81 


1.34 


38 


1.60 


1.29 


48 


1.55 


1.29 


72 


School Effort 


7.17 


1.69 


42 


7.42 


1.60 


36 


6.79 


1.67 


46 


7.07 


1.63 


64 


ilegattve Peer Influence 


1.80 


1.78 


40 


1.81 


1.86 


33 


1.89 


1.88 


44 


1.65 


1.92 


69 


Interpersonal Competency 


3.88 


1.00 


42 


3.91 


1.05 


38 


' 4.02 


1.09 


49 


4.05 


1.05 


72 


High Internal Control 


3.07 


1.30 


43 


3.30 


1.02 


38 


2.84 


1.08 


49 


2.90 


1.18 


72 


Prictlcal Knowledge 


8.57 


2.78 


42 


9.11 


2.89 


38 


9.53 


3.29 


49 


9.81 


2.86 


71 


Foiltlve Self-Esteen 


1.75 


4.08 


28 


1.54 


4.86 


24 


.99 


5.06 


29 


1.77 


4.93 


43 


Attachment to School 


3.81 


5.46 


42 


3.04 


3.35 


36 


1.02 


6.78 


47 


1.91 


5.45 


69 


Involvement 


-.53 


5.27 


40 


1.00 


5.80 


33 


1.67 


6.66 


46 


1.41 


6.27 


61 


Rebellious Autonomy 


1.58 


1.05 


43 


1.64 


1.12 


38 


1.85 


.92 


48 


1.81 


1.00 


72 



♦t-test for difference between treatment and control is statistically significant at the p<.05 level, indicating source 
of non-cqulvalence of groups at pretest. 

**t"te8t for difference between treatment and control is statistically significant at the p<.01 level, indicating source 
^ of non-equivalence of groups at pretest. 

Note . Refer to Chtipter 4 of the First SAES Interim Report for description of scale construction. Item number 32 was 
OBlttod from the ''Attachment to Parents" scale, item number 45 from the "Negative Peer Influence" scale. The sc.ile 
"High Internal Control" is not discussed in chapter 4. It includes the following three items from the pretest: 

—Luck Is more important than hard work. 

—Getting what I want has little or nothing to do with luck. 
Q —Every, time I try to get ahead, something or someone stops me. f,.^ 



Table 8 



Means and Standard Deviations 
on Fall '81 Pretest Measures — 
Muir Options Class 



Treatment Control 



Scale 


M 


SD 


N 




SD 


N 


Alienation 


1. 


,10 


1, 


.01 


51 


1.14 


1.14 


23 


Attachment to Parents 




46 


3, 


.13 


51 


.15 


2.10 


24 


Negative Belief (Non-Belief in Rules) 


1. 


,25 


1, 


.22 


48 


1.42 


1.26 


19 


School Effort 


7. 


U 


1, 


.61 


50 


7.33 


1.56 


24 


Negative Peer Influence 


1. 


,82 


2 


.07 


47 


1.17 


1.68 


23 


Interpersonal Competency 


3. 


,99 




.88 


50 


4.00 


1.00 


23 


High Internal Control 


3. 


,38 


1 


.02 


50 


3.09 


1.20 


23 


Practical Knowledge 


9. 


,49** 


2 


.51 


51 


7.69 


,2.67 


24 


Positive Self-Esteem 


1. 


,77 


3 


.39 


40 


2.08 


3.58 


21 


Attachment to School 


3. 


,84 


4 


.19 


47 


2.77 


3.29 


22 


Involvement 


-1. 


,11 


5 


.18:. 


46 - 


--.1.J4. 


_ 5 .69- 


. 23 - 


Rebellious Autonomy 


1, 


.68* 




.98 


49 


2.23 


.87 


22 



^^t-test for difference between treatment and control is statistically signifi- 
cant at the p<.05 level, indicating source of non-equivalence of groups at 
pretest. 

**t-test for difference between treatment and control is statistically signif- 
* icant at the p<.01 level, indicating source of non-equivalence of groups at 
pretest. 
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Table 9 



Treatment Control Group Comparisons on Several 
Measures of the Project Status Goal, 
"Decreasing Class Cutting" 



Treatment 


Class Cutting^ 


School Cutting 


Unexcused Absences 


and group 


M Sd N 


M 


Sd 




M Sd N 


ELIOT JUNIOR HIGH SCHOOL 


Options 












Treatment 


.61 1.19 56 


.43 


1.08 


56 


1.56* 3.15 64 


Control 


.36 .86 33 


.09 


.30 


32 


0.58 1.33 55. 


Youth Committee 












Treatment 


.71 1.11 42 


.29 


.74 


42 


0.43 1.07 54 


Control 


.50 1.05 72 


.36 


.92 


72 


0.99 2.04 86 


MR HIGH SCHOOL 



Options 

Treatment .91 .97 47 .57 1.16 47 1.42 2.83 55 

Control .93 1.00 40 .68 1.10 40 1.95 4.55 55 



Note: Due to the small number of non-equivalent controls (n=6) meaningful comparisons cannot be made between Youth 

Comittee members and controls at Muir High School, 

g < 
Class cutting and school cutting are individual items on the SAES student questionnaire. 

Pleasure based on school attendance records. 
*p<.05. 
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TriMitth'tit iinil Omirul Cnmii Compirlsnu); mi 
Si'ver.il MiMfinros uf tin- Pioiivi Si.iiiiii (lo.-il, 



TrcUftn'iii 
And f!r<)U|* 



'•I'll • U'j'Hi lnl 
''ii'.|'t'ii'h'ii^ 
:i Sd N 



TriMlnnii 
Co'.ilrol 

'iV'ilIt ''t't-:;*!!;!,' 

IriMltHMU 

Coiitrnl 

I 

^ 

O 

' Options 

Trciitrsi-nt 
Conirol 



J5 M 32 



.11 .32 35 
.25 62 



.09 .30 42 
.03 .16 39 



Im'II -I I'jkiH h'jl 

M Sd N 



l)tn}; ii.'ir 



|Ir||*r<'|iiirlcil J 
:;i>riiiii.s d('lliii|ii(>ni:y 
H Sd N 



•Fall 

r 

Suspc!ii!;i.'ins 
Sd N 



iMll D.HI, 

liX|ltllsIui1'i 

M Sd N 



I'XIOT JUNIUK IIIQ! SCIIDOI. 



.17 
.14 



.22 

.18 



48 
31 



.18 .19 3') 
.PJ .22 64 



.20 .2H /i8 
.20 .27 31 



.29 .29 37 
.27 .30 65 



.14 .22 49 
.10 .14 31 



.11 .19 39 
,13 .24 64 



0.0 0.0 64 
0.0 0.0 55 



i.O . 0.0 54 

i.n 0.0 Bfi 



0.0 0.0 

0.0 0.0 



p.O 0,0 
0.0 0.0 



64 
55 



INCH SCllOUL 



.20 
.16 



41 
37 



.34 ,31 40 
.33 .34 37 



.13 .18 41 
.OB .11 37 



,JD.0 0.0 
''O.O 0.0 



55 
55 



0.0 
0.0 



0.0 
0.0 



55 
55 



Note: Duo to Llw snail number of noi-cquiv.ilcnt controls {n»fi) meaningful comparisons cannot be nndc between 
youth f'-nniHiT turhors .md controls nc Miilr HIrIi Sclicol, 

lndividu.ll Iten on SAES student questionnaire. I 

litdivldunl-Ji'Vel !icudcnt sc.ilc 



.50 .5! 
.33 .47 



.40 .50 
.49 .51 



.35 .48 
.2? ,45 



Ilc.isurc bnscd on school records. 



5'J 



Table 11 





Treatment - Control Group 






Comparisons on a 






Measure of the Project Status 




1 


Goal, '^Decreasing Dropout Rate" 




Treatment 






arid Group 


Withdrawal from School 




M Sd 


N 


ELIOT JUNIOR HIGH SCHOOL 


Options 






Treatment 


0.13** 0.33 


64 


Control 


0.35 0.48 


55 


Youth Committee 






Treatment 


0.06 .23 


54 


Control 


0.13 .34 


86 


MUIR HIGH SCHOOL 


Options 






Treatment 


.09 .29 


55 


Control 


.20 .40 


55 



Note: Due to the small number of non-equivalent controls (n=6) meaningful comparisons 
cannot be made between Youth Committee members and controls at Muir High School. 

Withdrawals were collected from school records and include reasons such as transferred, 
graduated, involuntary, and voluntary. The reported mean combines involuntary and voluntary 
withdrawals . 

**p<.01. 
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T;il)U' 12 



Treatment anJ Control Comparisons on 
.SoviT.'il SAKS Ijiii'stionnnirc Mciiuiiri'S of Proioct StnUis' Objectives" 



Trciiinii'iit 



Alienation 
Sd 



I'ositive 
Self-esteem 
M Sil 



Att.icliini'nt 
to School 

I Sd N 



EI.IOT JllNIOI! illCII SraiOOl, 



' Fair 
Scliool Rules 
M SD 



Clear 
School liuies 
H SD N 



()|itii'ns 
Troalnient 
Cuiitrol 



.31* .25 47 
.A5 .23 28 



.72 
.72 



.17 
.19 



42 
26 



.71 .25 51 
.bl .24 32 



2.70** 1.81 
1.48 1.66 



53 
46 



3.42** 1.76 57 
1.77 1.87 47 



H 

N 
I 



V'lilli Commit tee 
Treatment 
Cmitrol 



.43 
.36 



.30 
.27 



34 
55 



.71 
.73 



.15 
.18 



31 
53 



.58 
.62 



.28 
.25 



37 
63 



2.09 
2.33 



1.66 
1.69 



46 
67 



2.83 1.69 46 
2.79 1.77 73 



miR Illdil SCHOOL 



Opt ions 
Tr(?atwt 
Oinrrnl 



.36 .27 35 
.31 .31 36 



.72 .19 32 
.73 .16 33 



.74 .23 44 
.75 .23 39 



2.51 
2.33 



1.64 47 
1.86 51 



T' 



Note: Due to snail number of non-cquivalcnt controls (n=6) meaningful comparisons cannot be made between Youth 
Cttnnilttee niemliers and controls at Huir Higii School. All measures are individual level student scales. 



^Scc section on objectives for more complete description. 
*P<.05 



**p<.01 



2.98 1.76 49 
2.63 2.01 54 
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Table 13 



Treatment and Control Group Comparisons on 
Several Measures of the Project STATUS Objective, 
"Iiicreaiic Atiadciiiic ArJiievi'iiR'HL" 



Treatment 
and Group 



I 

H 
vo 
u 
I 



Options 
Treatment 
Control 



Treatment 
Control 



Self -reported Grades^ Self-reported Reading^ 

Ability 

M Sd N H Sd N 



2.50 .84 54 
2.41 .61 32 



2.53 .83 43 
2.80 .80 71 



Fall GPA^ 
M Sd N 



ELIOT JUNIOR HIGH 



1.96** .79 
1.38 .80 



52 
26 



1.70 .79 40 
1.94 .87 69 



2.34 .66 53 
2.28 .62 41 



2.36 .77 46 
2.53 .85 80 



Spring GPA^ 
M Sd 



2.29 .61 . 54 

2.30 .68 36 



2.39 .74 50 
2.50 .89 74 



MUIR HIGH SCHOOL 



Options 
Treatment 
Control 



2.62 .74 47 
2.88 .85 40 



1.74 
1.92 



.80 
.76 



46 
37 



2.31 .91 48 
2.68 .91 43 



2.50 .87 47 
2.72 .92 44 



Note: Due to small number of non-equivalent controls (n=6) meaningful comparisons cannot be made between Youth Cominitt 
members and controls at Muir High School. 



An individual item on the SAES student questionnaire. 

Measure based on school records. 
*'>p<.01 



b, 
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Talilr |/i 



itid Croup 



)\lh*llt -.' ( rij'li.i'iis (*lt 



piiri'iiL-; 



lrr;|litrlit .iiid I'Miiiml CiniiiuriMiir: (III 
Srvi'nl iiiilcitiiii", t'f Tliciui'l ii'.il liili'n"it 



llrlirf in 

(idnvi'iuUiiKil I'uli'!' 



iiii'nu'rjiiiii.'il 
CniHpt'lniry 
« Sd N 



HLUIT JUNIOR lllllll SClinOL 



1.1 rti 
CiMniiutuiicy 
M Sd 



liit(*nt.il 
Coiitntl 
H Sd 



TreatniMU 
Contn'l 



.71 .29 i8 
.72 .:() 20 



.6i .26 ' 
.63 .25 



55 
33 



.65 .24 46 
.59 .25 25 



,m .18 46 
.73 .19 27 



K26 



.48 

,43 



43 
26 



.47 
.53 



.28 
.24 



46 
26 



.23 .21 
.19 .17 



Yoiitli 

Trc.itneiii 

H 

^ 



Options 
TrtM iir.cn E 
Control 



J\f\ ,26 34 
.Ml .32 -ifl 



M .29 37 
.56 .30 38 



.48** .31 42 
.64 .2«) 70 



.49 



.29 
.23 



46 
40 



.70 .22 33 
.64 .25 56 



MUIR HIGH SCIIOOl. 



.79 .22 33 
.80 .21 56 



U4I .50 32 
1.38 ;43 '55 



.59 .26 33 
.65 .26 38 



.76 .23 34 
.76 .22 38 



1.3*^ .32 
1.22 .41 



33 
38 



.40 
.46 



.26 
.24 



33 
55 



.44 .26 36 
.37 .29 36 



.23 .21 
.24 ,21 



.20 .20 
.16 .1) 



Sotu: DiK' to 'imall number of tinii-ri-iivnluni Vouih Cotimltttic controls (n''6) mdminRful coniparisdns cannot ht* nKulc 
betwrnTvoiith (^nniiu-o Mhm nnd rontrols .it Mulr lli;ji Scliool. All outconps are Indlvldiuil-Ievcl Htiitlonl scnios from 
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Project RETAIN, Chicago Board of Education: Evaluation Report 



J. St. John 



Abstract 

Responsive Education through Alternative Instructional Networks 
(Project RETAIN) was an alternative education project of the Chicago 
Board of Education sponsored primarily by a grant from the Office 
for Juvenile Justice and Del inquency Prevention (OJJDP) as part of 
its Program in Delinquency Prevention through Alternative Education. 
Operating in nine schools in the complex Chicago public school sys- 
tem, RETAIN sought to use individualized learning plans and the 
attention of special teachers to reduce problems of poor school 
attendance, disruptive behavior, and low student achievement. The 
evaluation provides no evidence that this project achieved its goals 
or the results sought by the OJJDP.,, This report describes the pro- 
ject and its evaluation. 



RETAIN (Responsive Education 
Through Alternative Instructional 
Networks) was a project undertaken 
by the Chicago Board of Education 
with support from the Office for 
Juvenile Justice and Del inquency 
Prevention (OJJDP) as part of that 
office's Program in Delinquency Pre- 
vention through Alternative Educa- 
tion (OJJDP, 1980). 

Project RETAIN Plans 

Goals 

RETAIN was aimed at the problems 
of poor attendance, disruptive 
behavior, and low achievement in 
Chicago public schools. Attendance 
problems included class-skipping and 
absenteeism in the high schools and 



This report supercedes an earlier 
report (St. John, 1982) on this pro- 
ject. It covers the entire two 
years of project operation. Donal d 
E. Rickert, Jr., analyzed the data 
for the second year of the project's 
operation, and wrote the section 
describing those results. 



absenteeism in the elementary 
school s . Disruptive behav ior 

included minor classroom or school 
disruption as well as serious 
assaults. Achievement problems were 
evident in the results of standard- 
ized tests. The project considered 
poor attendance and low achievement 
to be both individual- and school- 
level problems. 

Rationale 

The project's staff listed many 
causes of these problems, and 
divided them into five categories: 

(a) Parental and domestic causes, 

(b) school staff causes, (c) school 
administration causes, (d) community 
causes, and (e) student causes. 
Parental and domestic causes range 
from child abuse, to keeping chil- 
dren home to babysit, to an inabil- 
ity to meet the school health 
requirements for enrollment. School 
staff causes include ineffective and 
inappropriate teaching styles, cul- 
tural insensitivities and negative 
biases, and self-fulling prophecies. 
School administration causes range 
from attendance pol icies (locking 
school doors when the tardy bel 1 



-197- 



70 



RETAIN 



rings) to unfair disciplinary 
actions and the escalation of minor 
incidents into major episodes. Com- 
munity causes include discrimina- 
tion, unsafe streets, and lack of 
community support for the school. 
This lack of support occurs because 
the school "does not value community 
participation" and because the "com- 
munity does not view schools as a 
community structure that they are 
involved in influencing and cooper- 
ating with," Student causes might 
be economic concerns that conf 1 ict 
with school attendance, lack of 
motivation, not . valuing school, 
negative attitudes toward the 
school, poor peer relations, lack of 
social skills and self-discipline, 
low self-esteem, or lack of basic 
skills. 

Although RETAIN did not discuss 
delinquency as a problem or set the 
specific goal of delinquency preven- 
tion, the project's theory contains 
explanatory elements similar to 
those in Cloward and Ohlin's (1960) 
and Cohen's (1955) theories of del- 
inquency. 

Obiectives 

Although the project revised its 
list of objectives several t&nes 
during the 1981-82 school year, the 
list eventually produced resembled a 
list of interventions more than it 
resembled objectives (intermediate 
outcomes resulting from interven- 
tions, as we understand the term). 
In other words, the list of objec- 
tives produced may have resembled 
objectives of the "management by 
objectives" sort more than objec- 
tives in Program Development Evalua- 
tion sense. 

Although the project's theory 
specified the system's numerous con- 
tributions to the problems at hand, 
and the project's goals implied 
school-level outcomes, no school- 
level objectives were specified. 



Interventions as Planned and Modifi- 
cations 

First program year. During the 
first program year the RETAIN staff 
specified the following major inter- 
ventions: (a) increasing parental 
awareness, (b) linking families with 
needed services, (c) developing 
individual ized curricula, 
(d) advocacy, (e) peer group coun- 
seling, and (f) continued in-service 
training for . RETAIN teachers . These 
components were discribed in an ear- 
lier report (St. John, 1982), and 
were modifications of the interven- 
tions specified in the original pro- 
T^posal (Chicago Board of Education, 
1980). 

The primary focus of the project 
became the implementation of Indivi- 
dual Learning Plans (ILP's). The 
project's staff reported that ILP's 
developed . in -the- early, .stages .of 
implementation were not used effec"* 
tively. Accordingly, they decided 
that more in-service training for 
RETAIN teachers was needed to assist 
than witn developing and using this 
approach to education. Plans for 
the in-service training were carried 
out in September, 1981, just as 
school opened. 

Second program year . The list of 
RETAIN interventions continued to 
evolve during the second program 
year. By the end of the second pro- 
gram year, project managers listed 
the following maj or project compo- 
nents: (a) parental awareness 
activities, (b) efforts to establish 
linkages with Chicago Board of Edu- 
cation social service agencies and 
other community agencies and use 
these serv ice s , ( c ) indiv idual 
learning plans for attendance, 
behav ior , and achievement pr oblems , 

(d) career education (individualized 
and not for all participants), 

(e) computer-assisted instruction, 

(f) summer '82 RETAIN program. 
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(g) in-service training for RETAIN 
teachers > (h) staff intervention 
analysis meetings, and (i) evalua- 
tion tasks including student selec- 
tion, questionnaire administration, 
data collection, and case study 
reports. These components are 
described together with information 
on the degree to which they were 
implemented in a subsequent section 
of this report. 

Many of the changes from year to 
year were the result of attempts to 
reorganize plans according to Pro- 
gram Development Evaluation (PDE) 
definitions of interventions, objec- 
tives, strategies (see Gottfredson, 
1982) ; some changes reflect the 
amount of effort needed for success- 
ful implementation; and some reflect 
major changes in plans. These first 
year to second year changes can be 
grouped into two categories, 
(a) interventions dropped and added, 
and (b) shifts among FDE categories. 

Interventions dropped or added . 
Only the Peer Counseling component 
was dropped from overall project 
plans. . One factor contributing to 
the discontinuance of the component 
was scheduling. RETAIN students 
were scheduled for project services 
such that often only one or two stu- 
dents received services at a given 
time. Not all of the participants 
wanted to be in peer counseling 
groups and the chances of those who 
did being scheduled to attend RETAIN 
at the same time were slim. Alt- 
hough managers abandoned efforts to 
implement this component at each 
site, it remained the primary 
approach used by one RETAIN teacher. 
Seven of the remaining eight RETAIN 
teachers reported using something 
they called peer counsel ing with 
some students in their school • 

Managers added extensive plans 
for the successful implementation of 
their complex evaluation. The ela- 



boration of plans for evaluation was 
performed not because managers saw 
evaluation tasks as essential to 
their project, but because the 
national evaluators had observed 
evaluation tasks were not always 
sucessfully carried out in the first 
year (see St. John, 1982). It was 
hoped that careful planning would 
result in a stronger evaluation. 

Intervention Analysis Meetings 
were added by managers as a means of 
increasing the qual ity" and effec- 
tiveness of specific actions taken 
for individual participants. 

Shifts among PDE categories . In 
the first year. Advocacy (advocating 
for the rights and needs of partici- 
pants) was considered an interven- 
tion. However, a more detailed 
explanation revealed that this was a 
strategy used in the implementation 
of three components: Parental 
Awareness, Linking Families and Ser- 
vices, and ILP's. 

Computer-Assisted Instruction was 
listed as a strategy associated with 
the ILP's in the first year, but in 
the second year, was categorized as 
an intervention because of the con- 
sider abl e planning ef for t needed to 
make the computers operational. 

Career Awareness was shifted from 
the objective category to the inter- 
vention category because this was an 
activity planned for participants. 
This could be considered another of 
the ILP strategies, but since major 
efforts, especially in the summers, 
were devoted to this activity, man- 
agers decided to classify it as an 
intervention. 

The Implementing Organization 

Project RETAIN can best be under- 
stood in the larger context of the 
Chicago Board of Education (CBE) , 
the organization that was awarded 
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the grant to conduct the project. 
Members of Board are mayoral appoin*- 
tees. Through a long chain of com- 
mand , the Board oversaw implementa- 
tion of Project RETAIN (see the 
organization chart in Figure 1). 
The General Superintendent of 
Schools, appointed by the Board, has 
responsibility for implementing all 
school board pol icies. The project 
itself was under the Deputy Superin- 
tendent for Fiel d Serv ices , who 
delegated direct responsibil ities 
for overseeing implementation to the 
Director of Field services (DFS ) , 
Central. The project was placed in 
Field Services because of its plans 
to include non-academic services to 
participants and their families. 
The Department of Personnel hired a 
staff to work with the DFS to manage 
the project. The project management 
staff oversaw nine RETAIN teachers 
who worked in nine Chicago public 
schools. Eight of these teachers 
were selected early in the project 
according to Board guidelines, and 
one, whom a principal especially 
wanted in her school, was hired sev- 
eral weeks later. The third major 
division of the Board, Instructional 
Services, Government Funded Projects 
(GFP), was involved because the pro- 
ject was funded by the government; 
this division oversaw budgetary 
procedures. 

This arrangement was unusual for 
the Board: Most government grants 
are completely under the authority 
of the Deputy Superintendent of 
Instructional Serv ices , Government 
Funded Projects. The arrangement 
introduced unexpected elements into 
the project's f orcef ield as the two 
divisions of the Board developed 
working relationships. Both divi- 
sions had to be kept informed of the 
project's progress. Initially pro- 
ject managers expected that project 
planning and implementation would be 
reported to and approved by Field 
Services, and fiscal management mat- 
ters would be the purview of GFP. 



However , GFP assigned a program 
monitor to Project RETAIN in the 
second program year. 

It was al so unusual that the 
administrators overseeing implemen- 
tation and project teachers had 
direct contact. Ordinarily, project 
management would only have indirect 
contacts with teachers. Due to the 
nature of the project's activities , 
contact between project management 
and RETAIN teachers was arranged ; 
and formal structures* were main- 
tained- by communicating project 
plans and activities to intervening 
district superintendents and princi- 
„pals. 

CBE Environment 

Project RETAIN operated in a com- 
plex f orcef ield because of its 
af f iliation.with ^the. Board._ CBE _is 
currently under pressure from the 
Department of Justice to implement a 
satisfactory desegregation plan. 
During the 1981-82 school year, the 
courts and school board reached an 
agreement and the early stages of 
the desegregation plan went into 
effect. This public school system 
is one of the largest in the country 
and well over 50% of its students 
are minority. Declining enrollments 
for the last several years have 
forced school closings and teacher 
lay-offs. A sizeable budget deficit 
had to be immediately amel iorated 
because state laws make it illegal 
for public institutions to operate 
at a def icit . The Board of Educa- 
tion is under fire from parents and 
citizens groups, each with its own 
particular (and often conf 1 icting ) 
concerns. Very few days pass with- 
out news articles about the system 
and its problems. 

During the spring of 1981, a new 
General Superintendent of Schools 
was appointed by the Board of 
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Education from outside the system, 
causing uncertainty about steps that 
would be taken to solve the problems 
of the system. Many new administra- 
tors were appointed in 1982, includ- 
ing a new Deputy Superintendent for 
Field Services. This change had 
very little effect on RETAIN opera- 
tions. 

A critical incident . The General 
Superintendent sought additional 
funding for special programs as a 
means of addressing problems in the 
system. As part of this fund-seek- 
ing activity she called on officials 
at OJJDP to seek support for future 
delinquency prevention programs. 
During her visit, she was informed 
that her system currently had a 
large grant from the agency, and was 
told that both the evaluation and 
project activities needed strength- 
ening before more funds could be 
considered. OJJDP officials 

reported that she was apparently 
unaware of the project RETAIN — a 
circumstance that is not surprising 
because the long chain of command 
kept the project at some distance 
from her. 

On returning to Chicago the Geri^ 
eral Superintendent met with project 
managers, and subsequently appointed 
the GFP monitor . She al so appr oved 
the administration of the SAES stu- 
dent questionnaire from which the 
CBE had censored important items in 
the project's first year. CBE man- 
agers were invited to brief high- 
level CBE administrators about their 
project and its evaluation. For a 
time after these activities, the 
RETAIN managers' interest in PDE 
planning and project documentation 
increased. 



History of the Proiect 

The proposal for Project RETAIN 
was developed in part in response to 
public pressures on the school sys- 
tem to more adequately respond to 
the needs of troubled students and 
schools. The Board of Education 
assigned priority to alternative 
education and commissioned the Pro- 
gram Development Manager of the Cen- 
ter for Urban Education to develop 
such a program and secure funds • 
Prior to this proposal, one other 
unsuccessful attempt at funding had 
been made. 

The system had never before 
implemented an alternative education 
project closely resembling project 
RETAIN, but there have been other 
projects with similar goals. The 
ABC Project and the Family Guidance 
Center were both aimed at increasing 
student achievement and reducing 
absente* I -m and disruption. TAP-I 
was im lemeat- to reduce truancy in 
schools plagued by this problem, and 
worked with younger students identi- 
fiet as potential truants. Title I 
programs were planned to be imple- 
mented in eacl school, but implemen- 
tation did rot reach the final 
stages. TAP i was revitalized as 
part of th . desegregation plan in 
^98^-"^2. 

Proiect RETAIN Staffing and 
Resources 

Project RETAIN had four manage- 
ment positions, nine teachers, and £\ 
computer special ist . The Directoi* 
(not funded by the grant) had pri- 
mary responsibility for communicac-- 
ing with other CBE administrators. 
The Project Coordinator oversaw all 
project staff and the implementation 
of each aspect of the project. The 
coordinator worked directly with the 
teachers and kept the Director 
informed of progress and '.ssues. 
The Management Specialist weri pri- 
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marily responsible for fiscal 
management and evaluative tasks, and 
reported to both the Coordinator and 
Director. The Activity Specialist 
was responsible for working directly 
with the RETAIN teachers in develop- 
ing and implementing activities, and 
reported to the Coordinator. A Com- 
puter Specialist had responsibility 
for securing, installing, and pro- 
gramming a microcomputer system at 
the project schools and for provid- 
ing in-service training in the use 
of the computers. Teachers reported 
directly to the Activity Specialist. 
Total funding f or . two years of pro- 
ject RETAIN was $1.3 million, with 
most being allocated to personnel 
costs. (The two-year grant from 
OJJDP accounts for about $1.1 mil- 
lion; the balance was a CBE contri- 
bution. ) 

The Chicago school system pro- 
vided Project RETAIN with nine 
teachers (whose salaries were paid 
from grant funds) who were to imple- 
ment the project at each of nine 
sites, including three high schools 
and six elementary schools. The 
project managers conducted pre-ser- 
vice training in 1980-81 to acquaint 
those teachers with the goals and 
objectives of the project and to 
assist them in developing new 
approaches to the students who are 
often neglected by the system, and 
ongoing inservice training in 
1981-82 for continued assistance. 



The RETAIN Structure 

Project RETAIN served three clus- 
ters of three schools each: The 
Northside (School District 3), 
Southwest (School District 12) and 
Southeast (School District 19) com- 
munities. These clusters were cho- 
sen because they represent three 
different population groups from the 
city and are located in high crime 
neighborhoods. Each cluster is com- 
T^jsed of one high school and two 



feeder elementary schools (grades 
K-8) or middle schools. These 
schools were identified as having 
serious problems with one or all of 
the project^s target problems. 

Elementary school level . Grades 
7 and 8 are, for the most part, 
housed in elementary schools in the 
Chicago school system. These grades 
are organized into classes which 
stay together for the entire school 
day mostly in one classroom with one 
teacher. Some elementary school s 
have "walking" reading and math pro- 
grams > which mean the students 
change classrocrr.s to attend classes 
,,with special ty teachers. These 
schools are usually small with 2 or 
3 classes in each grade level . In 
some sob ool s , each grade level f ol- 
1 ows the same bas ic s ch edul e dur ing 
the day. 

This, highly structured jenyiron- 
ment helped structure the RETAIN 
project. At all elementary school 
sites, the RETAIN teacher had a room 
with a telephone set aside for 
RETAIN activities. The RETAIN 
teachers general ly visited other 
teachers each morning before school 
started. This was done to check on 
participant attendance , achievement , 
and classroom behavior from the pre- 
vious day. They returned to the 
RETAIN classroom to stand in the 
hall and greet students arriving for 
the school day . When a participant 
was not seen, other locations in the 
school were checked, and if the stu- 
dent could not be found, the RETAIN 
teacher contacted the parents by 
phone . In th is way , th e pr o j e c t 
teachers monitored the attendance of 
and had daily contact with partici- 
pants. 

Participants usually attended 
RETAIN in small groups of three or 
four during the school day . Small 
numbers allowed RETAIN teachers to 
work with students either in small 
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groups or individually and provide 
either academic instruction or coun- 
seling. The results of some stu- 
dent^s learning style inventories 
suggested that small group instruc- 
tion was appropriate for them; oth- 
ers needed individual attention or 
to work mostly on their own. RETAIN 
teachers attempted to schedule stu- 
dents for whom small groups were 
most appropriate at the same time 
period. But schedul ing constraints 
did not always allow for this — only 
certain time periods were available 
for attending RETAIN both because of 
student schedules and because RETAIN 
teachers had other project duties 
each day. RETAIN teachers had to 
have large blocks of time each week 
for making home visits and for 
attending every-other-week project 
staff meetings. Project staff 
reported that these constraints also 
resulted in a catch-as-catch-can 
schedule resulting in varying 
amounts of contact with RETAIN par- 
ticipants. 

High school level . Chicago high 
schools are large schools that 
include grades 9 through 12. Most 
of these schools have staggered 
start-up times for each class level. 
The daily schedule involves changing 
classes every 52 minutes for 7 to 10 
class periods each day. Students 
are not necessarily scheduled for 
instruction each class period. 

The school organization helped 
structure the high school level 
RETAIN project. Each RETAIN, teacher 
had a phone and classroom » or area 
of a classroom, reserved for RETAIN 
activities. General ly attendance 
was monitored by the having partici- 
pants come by the RETAIN classroom 
before reporting to their homeroom. 
Later in the day, some RETAIN teach- 
ers went on what they called 
"rounds." This activity involved 
checking the classrooms students 
were supposed to be in at that time 



period. Students who had not 
reported in the morning and. who were 
not in their classes were thought to 
be absent and thier parents were to 
be called. Students who were in 
attendance but not in their classes 
were to be counseled for class skip- 
ping. 

Participants were scheduled to 
attend RETAIN either during their 
study hall time or before school 
started in the mornings. The stu- 
dent's schedules determined when 
they Qould participate in RETAIN and 
how often during each week. If a 
student were selected and his or her 
schedule conflicted with the times 
the RETAIN teacher had available for 
classroom work, arrangements were 
sometimes made to change the stu- 
dent's schedule. When all routes 
were blocked the RETAIN teachers and 
participants worked out the best 
possibl e arrangement for early or 
late in the day contacts. RETAIN 
teachers tried to schedule students 
with similar problems and learning 
styles at the same time, but because 
of scheduling problems they were not 
always successful . 

Participants . The project plans 
specified that 15 students would 
receive direct services at each 
school. Students were to be chosen 
randomly from pools of referrals. 
School staff were asked to refer 
students with attendance , achieve- 
ment , or behavior problems (or any 
combinations of these problems) tp 
the RETAIN teacher. Parental con- 
sent was required for participation. 

Length of participation . Parti- 
cipants were to remain in the pro- 
ject until the RETAIN teachers 
judged that they no longer needed 
alternative education. The project 
did not specify how much contact 
each student would have with RETAIN, 
nor did it establish guidelines for 
the termination of participation. 
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Forcef ields 

Each teacher had responsibility 
to clear with the principal any 
alternative procedure or policy the 
project planned to implement in the 
school. For instance, in most pro- 
ject schools, the RETAIN teachers 
were relieved of hall monitoring 
responsibilities because the project 
required them to leave the campus to 
contact families. Regular responsi- 
— bilities in the school would have 
hindered the implementation of 
important project interventions. 
Project RETAIN managers reported 
that not all principals were suppor- 
tive of the project efforts in their 
schools. 

Probl ems in impl ementing some of 
the program strategies at some 
schools, including the advocacy role 
for RETAIN teachers and release time 
for RETAIN teachers' home visits, 
were never overcome. The principal 
at one of the schools reported that 
he suspected the teacher originally 
hired to implement RETAIN at his 
school was abusing the off-campus 
arrangements for making home visits. 
In the Fall 1981 school semester, he 
blocked implementation until the 
teacher was replaced with someone 
more to his liking in January 1982. 

This action (at Sheridan) created 
another problem. The project Activ- 
ity Specialist had to spend a great 
deal of time training the new staff 
member. Furthermore, the principal 
continued to disallow home visits. 

The special activities and status 
of the RETAIN teachers could have 
led to tense inter-teacher relations 
if the project's emphasis on helping 
students with special needs were not 
accepted by all teachers. Faculty 
contacts were maintained as a means 
of building support among peers for 
the project. 



Other school administrators and 
the Board supported the project, but 
the project managers spent much time 
working through bureaucratic proce- 
dures. 

Cross site variations . Extensive 
cross-site variations in the imple- 
mentation of the project resulted 
when the project caved in to princi- 
pal demands. At Bowen and Gage Park 
high schools and Bontemps Elementary 
School, the principals required that 
RETAIN teachers service more than 
the 15 selected participants. At 
Thorp elementary school the princi- 
pal used the RETAIN classroom as the 
^^site for an experimental in-school 
suspension project. The RETAIN 
teacher had responsibility for the 
suspended students and generally 
involved them in RETAIN activities. 
The necessity to be at school all 
day for this non-RETAIN activity 
prevented this RETAIN teacher from 
making home visits^ duFirig^ working 
hours. At Blaine Elementary School, 
the RETAIN teacher was often called 
upon to counsel non-RETAIN students 
referred to the office for discip- 
line reasons. Several RETAIN teach- 
ers reported that principals called 
on them to contact the f amil ies of 
non-participants because al 1 other 
attempts had been unsuccessful . At 
Bontemps, the RETAIN teacher 
expl ained that one or two students 
had be en pi aced in RETAIN so that 
the families could be contacted to 
give consent to assigning the stu- 
dent to the Special Education track. 

RETAIN teachers reported that the 
difference between the RETAIN parti- 
cipants and non-participant reci- 
pients of services was that no ILP's 
were developed for the non-partici- 
pants , and contacts were less regu- 
lar for most of these students. At 
most sites, principal demands were 
handled as additional responsibil i- 
ties. Managers reported these unan- 
ticipated variations did not harm 
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Sheridan Middle School was it neces- 
sary to drop a component. 

What Was Implemented ? 

The project did not participate 
in Program Development Evaluation 
(PDE) reporting procedures in the 
way we had intended. A management 
plan was not completed during the 
first PDE planning conference, nor 
was much systematic planning docu-- 
mented. In the first year, general 
repor ts of successes were of ten 
given. These reports usual ly tol d 
of necessary successes in procedural 
matters within the bureaucracy. 
Impl ementation in the school s was 
not documented, and so can not 
really be described. 

During the second program year 
project staff struggled with ' docu- 
menting programmatic efforts in 
their PDE plans. Although we 
attempted to clearly describe expec- 
tations, managers reported a lack of 
understanding of the expectations 
for planning and reporting. Several 
times they commented on the bulky 
plan that would result if they docu- 
mented project activities at each 
school . Such documentation would 
have been necessary because each 
component had some exceptional vari- 
ation at one or more of the sites. 
PDE pi ans were never devel oped ; 
other documents were substituted. 

Site visits were conducted as a 
means of gathering information about 
the project operations at the nine 
school s. All RETAIN teachers were 
interviewed about how they executed 
their understanding of the proj ect 
plans, a sample of ILP's were 
reviewed, and teachers were observed 
whil e working with students. Seven 
of the principals at the implementa- 
tion sites were interviewed. 

The following description of what 



RETAIN 

was implemented is based , then, on 
the documents furnished by RETAIN 
managers , observations at each 
school, and comments made during 
interviews with RETAIN staff. 

Parental awareness . Conferences 
with parents continued to be the 
primary method of making parents 
aware of the educational problems 
their children experienced and 
involving parents in the solution of 
these pr obi ems . The rol e of the 
RETAIN teacher in these" sessions was 
that of an advocate for the stu- 
dents. Many conf erences took pi ace 
in the homes of the students. 

At Nightengale Elementary School 
the RETAIN teacher organized a regu- 
lar weekly parents meeting. Parents 
at this school were assisting in the 
proj ect and three parents were at 
the school the day I visited. I 
also saw pictures in the RETAIN 
classroom of social events sponsored 
by the project for both participants 
at that school and for three RETAIN 
projects in nearby schools. Project 
managers indicated that the teacher 
was successful with the council 
because thv3 neighborhood had long- 
time residents who had always been 
active in school activities. At 
Blaine and Bontemps El ementary 
Schools, the RETAIN teachers had 
implemented individual counsel ing 
with several of the students' fami- 
lies in an effort to improve 
parent/child relationships. Some- 
times the teachers helped families 
begin counsel ing at local community 
mental health centers. RETAIN man- 
agers also devel oped three newslet- 
ters and sent them to the parents of 
participants. 

Linking f amil ies and services . 
The Chicago Board of Education is a 
city agency. This status could link 
the Board with other divisions of 
city government such as law enforce- 

t 
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ment, the judiciary, health and 
human services, and political bodies 
such as the City Council and the 
Mayor's Office. Project RETAIN is 
therefore potentially linked to 
these organizations through the 
Board. 

The Board of Education maintains 
many social and supplemental educa- 
tional services through its threo 
Pupil Service Centers, and by virtue 
of the project being a part of the 
Board of Education, these services 
were automatically linked. Project 
managers reported making arrange- 
ments for quicker processing 'of 
requests for services from these 
centers. 

RETAIN teachers and staff estab- 
lished relations with community ser- 
vice organizations and businesses 
near some of their schools. Three 
schools reportedly established 
informal RETAIN Parents Councils 
that included members from community 
organizations. RETAIN managers also 
acquired a list of service organiza- 
tions categorized by geographic area 
and type of service and distributed 
the lists to the RETAIN teachers. 

Although project managers pro- 
V ided 1 i t tl e in th e way of gui de- 
lines for determining whether or not 
to refer families for services, the 
teachers apparently followed their 
hunches about family circumstances 
and consul ted other people acq- 
uainted with the students. Clearly 
some decisions needed 1 ittle gui- 
dance — one teacher reported she had 
referred a family to the local men- 
tal health center because t:he father 
v/as an alcoholic who .^J^used his 
child. 

Individualized Learning PI ans 
(ILP's) . RETAIN teachers gathered 
social and academic his tories for 
the participants and devised indivi- 
dual pi ans for ach ievement, atten- 
dance, and behavioral objectives 
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based on this information. The 
teachers were also expected to det- 
ermine their students' individual 
learning styles and plan teaching 
methods that matched student learn- 
ing styles. Once these data were 
gathered, students and teachers were 
expected to develop a contract that 
specified long term goals, short 
term objectives, the activities 
planned for reaching the objectives, 
arid the means of regular evaluation 
of the students progress. For exam- 
ple, a student who had •been chroni- 
cally .absent might have as a goal to 
attend school every day he or she is 
not ill. The short-term objectives 
might be to first begin attending 
three days a week, then four, and 
f inal ly f ive , with at tendance 
reported by the students' regular 
classroom teachers as the means of 
evaluating whether or not the objec- 
tives were reached. 

Individual ized"^ pl'an's * "were" ~to 
cover any individual counseling 
needs with which RETAIN teachers 
could help. The plans were also to 
specify individual career interests, 
plans for increasing awareness of 
jobs in fields of interest, instruc- 
tion needed for particular jobs, any 
job seeking-skills that may be 
needed, and appropriate job place- 
ments for eligible students. Compu- 
ter-assisted instruction was planned 
as one aid to implementation of 
learning plans. 

The project managers encountered 
difficulties in their attempts to 
standardize the ILP recording format 
across schools . Despite the manag- 
ers' honest efforts to devise a com- 
mon format by the end of the year 
there were as many recording formats 
as there were RETAIN teachers. The 
Project Coordinator reported that 
this was because each teacher f el t 
more at ease with his or her own 
recording format. 
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RETAIN teachers varied in the 
degree of specificity of both long- 
term goals and short-term objec- 
tives. Several reported confusion 
over these details. Despite these 
problems, each current participant 
at each school had a file folder 
containing records of goals and 
objectives contracted for, and lists 
of actions taken by each teacher, 
dates the action was taken, and some 
report of results of the actions. 

Students reported they knew about 
their plans and contracts; several 
could state some of their goals and 
the reasons they thought working to 
achieve the goals would be benefi- 
cial. Students reported they were 
aware of most of the actions, taken 
on their behalf by the RETAIN teach- 
ers, and many reported they appreci- 
ated the caring and help re- 
ceived — even when the help involved 
the dreaded parent contacts. 

Career awareness ^ Proj ect staff 
reported that records of specific 
career awareness activities for the 
individual students appeared on 
their ILP's. Some of the approaches 
included the use of CVIS available 
on the Board of Education's downtown 
computer, speakers, visits to job 
sites, study of occupation-specific 
preparation units before visiting 
sites, help vith job seeking skills, 
and the 'Vorld of Work" component of 
the 1981 summer program. 

Computer-Assisted Instruct ion . 

This component met with obstacle 
after obstacle and> although planned 
to begin in the first year of imple- 
mentation, did not get off the 
ground until the fall of the second 
year. The first unanticipated 
obstacle was that the Board of Edu- 
cation guidel ines required that the 
purchase of the equipment be con- 
ducted through a competitive bidding 
process. This unanticipated 

requirement delayed obtaining the 



equipment. Later, arrangements for 
electrical outlets and overnight 
security slowed down instal 1 ing the 
equipment. Then dif f icul ties 

acquiring software (computer pro- 
grams) were encountered when project 
managers found thaC< the Board of 
Education al so had guidel ines for 
what could and could not be 
installed on the computers. The 
computer expert on the management 
team developed a few small programs 
for the computers so teachers and 
students could begin* using the 
equipment, and he conducted on-site 
workshops in the use of the 
machines. The software was eventu- 
j^ally acquired and installed, but one 
of the schools had its computer sto- 
len. Parents at this school were 
working to replace the computer when 
I visited the school in March. 

At some .schools the RETAIN teach- 
ers regularly used the computers to 
give additional instruction in math 
and English. One student reported 
that the machine was fun to use 
because it divided complex problems 
into little steps and immediately 
reported to the student whether or 
not each step had been properly exe- 
cuted. Another student had begun 
writing simple programs that made 
the computer print out geometric 
designs. The computer kept indivi- 
dual logs of student use that 
included amomit v f time, units com- 
pleted, and degree of mastery in 
each unit. Teachers yho used the 
computers recorded the computer log 
information on studencs ILP's. 

In the beginning of the school 
year project managers had plaim<jd 
for students to be trrasported to 
DePaul University for additional 
computer instruction. These plans 
had to be abandoned because of the 
high fees that would have been 
charged , large amounts of time that 
woul d be ueeded to transpor t --i u- 
dents to the University and ck 
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again, and because University 
personnel expressed some reserva- 
tions about having groups of young 
students using their facilities. 

Summer program . Dur ing the sum- 
mer of 1981, the project implemented 
a special program to work with all 
RETAIN students for the entire day 
at the nine school sites. The sum- 
mer program emphasized career educa- 
tion and remedial instruction. They 
had hoped to employ the RETAIN high 
school students as tutors for the 
elementary students with CETA fund- 
ing but were unable to do so. 

Early in the 1981-82 year , pro- 
ject managers had not decided 
whether they would implement the 
simimer program for a second year. 
The final decision rested on whether 
or not the Board of Education plan- 
ned to have any schools open for 
summer school. We do not know if a 
simimer program was implemented in 
the summer of 1982. 

In-service draining » The forego- 
ing list of cdjnponents planned as 
direct services to students is indi- 
cative of the variety of skills the 
on-site implementers must have had 
in order to successfully serve stu- 
dents as planned. Because teachers 
selected to implement the proj ect 
had, for the most part, been class- 
room teachers, project managers felt 
that many hours of in-service train- 
ing would be required. Project man- 
agers began the training by acq- 
uainting the teaching staff with the 
project goals and planned interven- 
tions prior to student selection for 
participation in the first program 
year. They also thought in-service 
training should occur periodically 
throughout the implementation period 
so teachers could acquire skills and 
information needed to (a ) implement 
the career development curriculum, 

(b) improve student reading skills, 

(c) . use computers as an instruc- 
tional aid, (c) counsel students. 



and (d) determine individual stu- 
dents'' optimal learning styles.. 
Teachers were also to learn about 
family services available through 
the Board of Education and other 
community agencies. 

For the most part, . project manag- 
ers did not make specific plans for 
specific types of in-service train- 
ing after the initial pre-service 
training. The management style was 
one of informally surveying teachers 
to find out what areas of respon- 
siblity they thought they could use 
some additional training in> then 
trying to identify resources for » 
„ delivering the training. Often the 
managers learned of particular work- 
shops that were planned elsewhere 
and spontaneously arranged for 
release time. This lack of specif- 
icity became clear when project manr 
agers asked Polaris, the technical ' 
assistance contractor for the OJJDP 
initiative, to come and train teach- 
ers in effective peer counseling 
methods. The. impression of the TA 
contractors was that the project': 
managers were unclear about how the 
proposed training in peer counseling 
fit into their project plans. Alt- 
hough peer counseling was initially 
planned as a component of the pro- 
ject, it had, by that time, been 
abandoned for numerous reasons at; 
all sites. The project managers'' 
decision to seek training in peer 
counseling when they did was somew- 
hat surprising. 

Intervention analysis meetings . 
By January of 1982, project managers 
added a new component for their 
teachers called Intervention Analy- 
sis Meetings. This was planned to 
be a means of helping ter^chers to 
solve problems through the presenta- 
tion of case studies at staff meet- 
ings. Other members of the staff 
were expected to analyze the case 
based on the presented materials and 
suggest possible interventions. We 
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httve boon unable to dctermino 
vhothor this component was imple- 
muntod* 

£,\Lfl]MflUqiL> activities. At the 
AuQuttt 1982 planning conference i 
project managers developed extensive 
plans for carrying out each ♦:a8k of 
their evaluation* They had been 
unable to gain approval in the first 
year to administer the SAES questi- 
onnaire containing the self-reported 
delinquency items* The Chicago 
Board of Educations censoring these 
items from the questionnaire 
resul ted in no assessment of the 
primary result sought by the sponsor 
of this delinquency prevention pro- 
ject because the Board also blocked 
attempts to acquire official delin- 
quency data from police or court 
sources* ^tanagers reported that 
failure in this area was primarily 
due to a fear on the part of admin- 
istrators and principals that 
parents would object or cause prob- 
lems for principals and the school 
system if they knew the project was 
part of a delinquency prevention 
initiative^ or knew about the sensi- 
tive nature of some of the questions 
being asked of their children* 

In the second year, project man- 
agers set a timeline for contacting 
high-level administrators to gain 
support for the questionnaire, but 
their plans did not include any 
strategies for helping the adminis- 
tration cope with irate parents 
should this unlikely event occur* 
As it turned out, the matter became 
moot when the questionnaire was 
brought to the attention of the Gen- 
eral Superintendent after her meet- 
ing with OJJDP officials* We wrote 
to the Superintendent requesting 
approval of the standard student 
questionnaire for administration in 
Chicago. The Superintendent reluc- 
tantly approved the questionnaire* 

Overall school response rates on 
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the questionnaires were low in the 
first year's administration* Manag- 
ers indicated that principals saw 
the questionnaire as a disruption to 
their school and as having little or 
no value* Managers therefore m^de 
plans for helping principals by 
developing special reports on some 
of the questions and reviewing the 
results they thought principals 
would find to be useful information* 
These tasks were executed, but it 
had little effect on the response 
rates in the second year* 

Another plaguing problem was the 
likely breakdown of the experimental 
design at some schools* Post-ran- 
domization checks shewed that ran- 
domization had failed in several 
schools in the projects first year* 
Managers decided to resolve this 
problem by using a random number 
table to make the selections at the 
RETAIN central office* This proved 
to be an unsatisfactory^ solution "to 
the problem when evidence of another 
breakdown in randomization began to 
accrue* 

Because the project also had 
school-level goal s, plans were made 
for reporting school level data to 
the national evaluation team* The 
project identified several control 
schools to use in the comparative 
analysis, and decided to administer 
the questionnaire in one of these 
schools* These plans were not fully 
executed* No school-level data were 
reported for any of the schools* 

Problems also surrounded indivi- 
dual-level data collection activi- 
ties* During the first reporting 
period of the project's second year 
staff at t'ii^ schools who collected 
and reported data on the treatment 
and control groups misunderstood 
some printed instructions* At the 
end of the first reporting period 
this problem was discovered, and by 
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the last data reporting period codes 
were being properly used and all 
instructions were followed. 

Other components . Managers gave 
oral reports about special activi- 
ties which some of the RETAIN teach- 
ers implemented at their schools. 
At Blaine Elementary School, the 
RETAIN participants wrote and per- 
formed a play for their school. The 
Northside cluster took a field trip 
to Evans ton for the experience of 
attending a professional perfor- 
mance. 

Effectiveness 

Evolution of Evaluation Design 

First Year . The project managers 
established procedures for random 
selection of students at each 
school. More than 15 students were 
referred at each school; thus selec- 
tions had to be made in some fash- 
ion. Randomization was agreed upon 
because it seemed administratively 
simplest. Principals, counselors, 
and RETAIN teachers then made the 
selections by drawing names of stu- 
dents referred to RETAIN out of 
hats. Lists of RETAIN participants, 
control students, and waiting stu- 
dents were developed. The project 
reported that there were only a few 
students in the nine schools who 
were placed in the project by any 
method other than drawing names from 
hats. N 

This method of randomization did 
not produce equivalent groups of 
students. Post randomization checks 
revealed significant differences in 
achievement (standardized test 
scores and grades) for the 



1. Alas, they were told to "draw 
names from a hat" by the evaluation 
field worker initially assigned to 
work with the project. 



comparison groups at two schools. 
In one of these schools, the control 
group contained a high proportion of 
special education students. Random- 
ization may also have broken down in 
a third school . A significant dif- 
ference in the number of siblings 
reported by treatment and control 
Cuudents was found in that school. 
As a result, project managers com- 
mitted themselves to conduct the 
randomization process and use a ran- 
dom numbers table for the selections 
in the second year , and the place- 
ment of any student in the pr oj ect 
by any other means was to be docu- 
mented. 

Second year . Although we urged 
that randomization be conducted at 
Johns Hopkins , proj ect managers 
demurred. Several indications imply 
that the experimental design al so 
broke down in the second project 
year . Some students who had parti- 
cipated in the program previously 
were reenrolled in the program. 
And, project managers used random 
number tables to select new partici- 
pants for the second year before 
obtaining parental permission for 
participation in RETAIN and the 
evaluation. Many students selected 
for the control group were then 
excluded from the study when paren- 
tal permission could not be 
obtained. We experienced considera- 
ble difficulty in determining 
exactly what happened to the design. 

When we discovered that implemen- 
tation had been delayed in one of 
the project^s schools (Sheridan), we 
again reque sted to conduct randomi- 
zation at Johns Hopkins. This 
request generated some ill will, but 
was acceded to. A one-semester true 
experiment was definitely conducted 
at Sheridan, although project imple- 
mentation was weak in this school. 

Both years . As with all projects 
in the Alternative Education Pro- 
gram, project staff were expected to 
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take all necessary steps to secure 
information pertinent to the evalua- 
tion from school and other archival 
records, and to administer the 
School Action Effectiveness Study 
student questionnaire each spring. 
Archival data were to be collected 
for RETAIN participants and control 
group youths, and questionnaires 
were to be administered to these 
youths and to probability samples of 
students from participating schools. 
Survey administration was to be 
guided by administration manuals 
supplied to the project. 

Results First Program Year 

The clear evidence of a breakdown 
in randomization in two schools, and 
the possibility that it failed in a 
third, made us wary of assuming that 
randomization was implemented in the 
remaining schools. Because we did 
not supervise the randomizatioa, we 
have no direct knowledge of the 
procedures used. Despite these wor- 
ries. Some significant differences 
between RETAIN and control students 
were found and merit discussion. 
Interpretation should be tempered by 
awareness that (a) randomization may 
have broken down in several addi- 
tional school s , if not in all of 
them, and (b ) interventions were 
implemented in limited forms, and for 
only a short period of time. 

No statistically significant dif- 
ferences were found on any cognitive 
variables , after the two school s 
where evidence of non-random assign- 
ment involved cognitive variables 
were excluded from analyses. Mea- 
sures of delinquent behavior were 
not avail able for analysis : Nearly 
all sel f-repor ted del inquency items 
were censored from the questionnaire 
by the Chicago Board of Education, 
and the Board refused to seek offi- 
cial delinquency data. 



In one project high school 
(Bov/en) RETAIN students were absent 
significantly more often than con- 
trol students (6.11 days vs. 2.67 
days on average , respectively , 
p<.01). In Blaine Elementary 

School, Interpersonal Competency 
scores were higher for RETAIN than 
for control students (4.07 vs. 3.21, 
p<.05) (This is the school in 
which RETAIN and control students 
differed in number of sibl ings . ) 
Several differences between RETAIN 
and control students were found in 
Bontemps El ementary School . RETAIN 
students were significantly lower in 
Interpersonal Competency than the 
controls (2.96 vs. 4.30, p<.05), 
higher in Rebellious Autonomy (2.21 
vs. 1.37, p<.05), higher in self-re- 
ported suspensions (.76 vs. .28 per 
student in the past year, p<.05) . 
Finally, in Thorp Elementary, RETAIN 
students scored higher than control 
students in Attachment to School 
(4.19 vs. -3.69, p<.01). - 

Thcise interim results were inter- 
preted with caution for several rea- 
sons. First, we were uncertain that 
the RETAIN and control groups were 
truly randomly equivalent. Second, 
the available data were dredged to 
detect significant differences , and 
when many significance tests are 
conducted a few may be "significant" 
by chance . Third, the pr oj ect 
implemented its interventions in 
weak form. Although detail s were 
sketchy , ILP's were apparently not 
regularly updated, the parent 



2. Descriptions of the measures 
used in the School Action Effective- 
ness study during year one may be 
found in the first interim .report 
(Gottf redson, 1982) and descriptions 
of the si ightly modified measures 
used during year two may be found in 
the overview section of the second 
interim report (Gottf redson, Gott- 
f redson, & Cook, 1983). 
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component did not go as expected, 
and the intervention was brief. 
Questionnaires were administered 
only 8 to 10 weeks after implementa- 
tion of the project began. These 
results apply only to the interven- 
tions as implemented, not to the 
interventions intended. 



Second Year Resul t5; 

During the 1981-82 school year, 
163 students in Project RETAIN 
nine target schools were identified 
as being control students , and 145 
were identif ied as treatment stu- 
dents • Table 1 shows the number of 
treatment and control students in 
each school. Usable pre-tr eatment 
data f roi:; records for the 1980-81 
school year were provided on stu- 
dents' gender, age, fourth quarter 
English grades, Iowa Test of Basic 
Skills (ITBS) scores in reading and 
math (for elementary school students 
only), number of high school credits 
earned (for high school students 
only), and suspensions. An unfortu- 
nate amount of data were mis- 
sing — ranging from. 21% missing for 
Engl ish grades to 35% missing for 
suspension data. As Table 2 shows, 
there was considerable variation 
across schools on the degree of 
missing data. 

Outcome data from records for the 
1981-82 schools year were provided 
for attendance, suspensions, Eng- 
lish, math, science and social stu- 
dies grades, and withdrawal from 
school. The amount of missing or 
incomplete data ranged from 0% for 
withdrawals to 60% for attendance. 
As was the case for pre- treatment 
data, there was considerable varia- 
tion across schools (see Table 3). 

Completed or partially completed 
Student Questionnaires (administered 
Spring, 1982) were returned for 65% 
of the treatment students and 52% of 
the control students. Survey 



administration was poorly imple- 
mented in several school s. Propor- 
tions of questionnaire data missing 
for each of the proj ect's schools 
are shown in Table 4. Tlie question- 
naire, although a post-treatment 
measurement instrument, contains 
measures on several measures, such 
as age , sex, race and parental edu- 
cation that are considered to be 
background measures or measures of 
stable personal characteristics that 
could probably not be affected by a 
treatment . They are thus useful in 
assessing the pre-tr eatment equiva- 
lence of treatment and comparison 
groups. 

Analysis proceeded in several 
steps. First, treatment and control 
students were compared on available 
pre-treatment and Student Question- 
naire background variables. These 
comparisons were performed at the 
project level (i.e. all project 
treatment students vs. all proj ect 
control s ) and were attempted at the 
school level (i.e. , treatment vs . 
controls in each of the target 
school s) . Comparisons at the target 
school level are largely uninterpre- 
t abl e due to smal 1 N' s and many 
missing data. 

-If the project had substantial 
effects, this should show up in the 
project-level comparisons, where, 
despite the unknown bias due to 
missing data, the N's are large 
enough to allow a reasonable chance 
of detecting statistically signifi- 
cant differences if they exist. 
Comparisons of treatment and control 
groups were made for outcome mea- 
sures from school records as well as 
outcomes measured by the Student 
Questionnaire. For outcomes for 
which the difference between treat- 
ment and control students was signi- 
ficant, subsequent analyses examined 
the data to determine whether treat- 
ment by school interactions were 
present. A significant interaction 
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would suggest that the project was 
more effective in one school or sec 
of schools than in others. 

Because the randomization process 
had broken down, for treatment-con- 
trol group comparisons in which ini- 
tial analyses found significant dif- 
ferences, statistical controls were 
applied in a stepwise regression 
analysis to "control" for pre-exist- 
ing differences between groups. 

. Post randomization checks . When 
al 1 proj ect treatment students were 
compared to all control students on 
avail able pre- treatment and back- 
ground data, no significant differ- 
ences were found (see Table 5). 
Furthermore, differences between the 
two groups were small in comparison 
to the individual differences within 
groups (compare the differences bet- 
ween means in Table 5 with the stan- 
dard deviations, which are indica- 
tors of the extent of individual 
differences ) . Comparisons on pre- 
treatment and background variables 
were also performed school -by- 
school. This time, several signifi- 
cant and near signif icani: differ- 
ences appeared, despite the small 
N's (see Table 6). Because randomi- 
zation was to occur at the school 
level , this set of results impl ies 
that the evaluation must be regarded 
as involving a non-equivalent con- 
trol group design , because randomi- 
zation was not effective. 

Outcomes for treatment and com - 
parison groups . Post-intervention 
comparisons between treatment and 
control group students are summar- 
ized in Tabl es 7 and 8 . These are 
simple raw comparisons, with no sta- 
tistical controls . Three outcomes 
favored the treatment group. Tliese 
were: dropout (p<.01) , practical 
knowledge (p<.05), and self-reported 
employment (p<.01) . One result 
favored the control group — parental 
emphasis on education. When statis- 



tical controls were applied in a 
regression model, ^ no significant 
differences between groups remained, 
except for sel.c-repor ted employment 
and dropout in school 1240 only. It 
turns out tha*. the difference bet- 
ween treatment and control students 
on dropout rate in school 1240 is 
due to a very high rate among con- 
trol students.^ These results may be 
interpreted as suggesting that the 
observed differences between RETAIN 
and comparison youths can plausibly 
be attributed to pre-existing dif- 
ferences; they may not be attributed 
to the program's influence with any 
conf idence. 

Discussion 

The present results are subject 
to several important 1 imitations. 
The most important limitation is the 
overall poor quality of the data. 
Missing and incomplete data, and the 
variations across schools in quality 
may introduce bias of an unknown 



3. A control variable is a variable 
that is correlated with the outcome 
variable of interest which is some- 
times introduced into an analysis to 
adjust for pre-existing differences 
between groups when the groups are 
not randomly equivalent. The fol- 
lowing variables were controls for 
the regression of dropout rate on 
treatment: age , suspensions in 
1980-81 , and absences in 1980-81 . 
For the regression parental emphasis 
on education on treatment , the fol- 
lowing variables were utilized as 
control s : parental education, dummy 
for school 4440, and the dummy for 
race (l=white, O=other). For the 
regression of practical knowledge on 
treatment, these variables were used 
as controls: sex, dummy for race 
(l=white, O=others) and the dummy 
for school 1240. The control varia- 
bles used for the regression of 
self-reported employment were : sex 
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nature into statistical analyses. 
It also weakens analyses by decreas- 
ing statistical power, i.e., 
decreasing the chance that differ- 
ences that may exist will be 
detected by a statistical test for 
significance. A second limitation 
is the non-equivalence of groups 
prior to treatment (See Table 5 ) . 
Groups were to have been randomly 
assigned to treatment and control 
conditions, but this was either not 
accomplished or was unsuccessful. 
Finally, we judge the implementation 
as not particularly a strong one, 
and therefore statistical results 
should probably be interpreted "as 
representing the resul ts for a 
rather weak set of interventions. 



Results of First Program Year 

Despite the limitations, the 
implications of the first year 
results for the project were sev- 
eral . Earl ier (St. John, 1982) we 
listed several suggestions: 
(a) Steps should have been taken to 
assure that randomization was imple- 
mented, (b) The project should have 
examined activities at Bowen to det- 
ermine whether any aspects of RETAIN 
activities contributed to stude^*" 
non-attendance. (c) The project 
should have examined its activit'-<cO 
at Blaine to learn whether project 
activities that contributed 
interpersonal competency could be 
identified and put into practice 



and the dummies for schools 1240, 
6180 and 5090. 

4. For control students in school 
1240, the sample mean for dropout 
rate is .55 with an SD of .51 
(N=18). For the treatment group the 
sample mean is .13 (SD=.35, N=15). 
See Table 7 for the sample means and 
standard deviations for the entire 
sample (i.e. all treatment vs. all 
control students) . 



elsewhere. (d) The activities at 
Bonteiaps should be have been care- 
fully scrutinized to s:: ' if changes 
coul d have been made tc '^void what 
appeared to be negative 'i* :ects on 
interpersonal competency , ^11 ious 
autonomy, and possibly ?u^p». ->sions. 
(e) The activities at Thor^: -hould 
have been examined to learn w»-':-r?"?r 
project activities that contr iL. »'.i. 
to student attachment to schoe.. 
coul d ha^v. been identified and pvi* 
into pracc^ . elsewhere. 

Possibly J of course, tacso 
resul ts were a-vtributabl e to 
breakdown of r " ndotnization. Thsr.^- 
,,fore, we suggs:-" e that the project 
carefully reviav hov/ students v^^re 
selected for participation in 
RETAIN, and consider whether these 
results may Lave been due to some 
artifact of the selection procedure. 
Although project managers made some 
attempt to improve their_ evaluation, 
there is little evidence that any of 
these suggestions were followed up 
with much zeal • 



Se cond P rogram Year Results 

Taken together, the results for 
th second y ear pr ov ide little ev i- 
dence that RETAIN accomplished 
either its own goals or the results 
soii:r;ht by OJJDP. This outcome may 
be the result of (a) a weak evalua- 
tion, or (b) a weak program. 

PDE and The Proiect 

Project RETAIN managers did not 
fully participate in a Program 
Development Evaluation of thri kind 
we had envisioned would be imple- 
mented. They were cooper a. ,ive in 
the summative aspects of evaluatio**, 
el ecting to attempt impl ementation 
of a true experimental design and to 
report school record data on indivi- 
duals in a timely and frequent fash- 
ion. They did uot, however, furnis- 
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systematic implementation data to us 
as requested. We do not fully 
understand the reasons we were una- 
ble to obtain iir.pl ementation data, 
the reasons for the extensive miss- 
ing data problems, or the reasons 
for the failure of randomization. 

Goals . Project managers partici- 
pating in the first PDE planning 
workshop were clear about the prob- 
lems th e pr o j e c t wa s de s ign e d to 
address . Therefore they were cl ear 
about the goal s of the proj ect and 
the target population that was 
appropriate for the direct services 
component . 

Theory and objectives . The pro- 
ject's rationale pointed to many 
sources of the problems RETAIN was 
intended to address, and some of the 
implied corrective activities were 
incorporated into planned activi- 
ties. For the most part, the plans 
developed failed to clearly specify 
the intervening outcomes that would 
have to be achieved. We sensed some 
reluctance on the part of project 
managers to make clear statements in 
this area. For example, although 
managers stated that one cause of 
the problems that RETAIN was to 
address was ravaged self-esteem, 
they would not state this obvious 



5. It may be of some significance 
that, after extensive discussion of 
the advantages and disadvantages of 
al ternative evaluation designs at 
the January 1981 workshop where the 
experimental ' approach was decided 
upon, the reason project managers 
chose to implement randomization 
rather than al ternatives (e.g. 
matched control groups, regression 
discontinuity, time series) was that 
they judged it to be administra- 
tively simpler to accomplish. In 
actual practice, maintaining the 
randomized treatment-control group 
design was not simple. 

-21 



outcome as an objective of their 
interventions because they said they 
did not want to be held accountable 
for raising self-esteem, a rela- 
tively stable characteristic that is 
dif f icul t to change . 

Proj ect managers appeared some- 
times to use the word "obj ective, " 
in ways other than that intended in 
the PDE me thod . In o ther pi anning 
models, "objective" is often used 
synonomously with a process or 
activity. PDE requires abandoning 
that usage and learning a more 
research oriented meaning of objec- 
tive. Activities or processes are 
called "interventions" in the lan- 
"guage of PDE. 

The Match between Theory and Inter - 
vent ions 

The project's theory described 
the contributi-" ijs of students, 
parents, community, and school to 
the problems that students experi- 
ence. It follows, therefore, that 
objectives and interventions should 
be aimed at system chn-iges, with at 
1 east as much vigor as the indivi- 
dual focus. Although plans ini- 
tially included several intermediate 
outcomes and project activitit^s that 
were related to chan^^i^s in the sys- 
tem, nearly al 1 were dropped. Only 
at Thorp vas there an experiment 
with an in-school suspension project 
initiated by the principal. Except 
insofar as the use of ILP's and the 
attendance methodology were viewed 
as a pilot or demonstration of these 
techniques, RETAIN may then be char- 
acterized as a project aimed at 
assisting individual s in jar si sting 
despi te the system. If the pro- 
j ect's theory was correct, RETAIN 
was probably not abl e to achieve 
dramatic effects when no system 
changes were made. The focus on 15 
individual s may not be a strong 
enough interveiition to affect a 
whole school — even a small one. 
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School-level , From the outset 
project staff stated school-level 
goals of increasing attendance and 
reducing disciplinary incidents. 
Although the school-level interven- 
tions were dropped, the goals were 
clung to as if the project'^s success 
were riding on having reached these 
goals. 

The project managers explained 
this was because Project RETAIN was 
itself an experiment with new- ppl.i- 
cies and procedures. They;^ argued 
that implementation of the-' pro j ect 
was therefore evidence that the 
goals were met. * As for creating 
broad changes in school policies or 
practices , the staff reported that 
th is woul d have taken years of 
effort, involving every level of the 
Board in planning and approving new 
pol icies. Furthermore , they were 
not policy makers, but project 
implementers responsible for a pro- 
ject serving 15 students at each 
school. They did indicate that the 
project was being carefully watched 
by administrators and that if the 
al ternative pol icies used in the 
proj ect appeared to be successful , 
pol icy makers would consider adop- 
tion in the system. 

Conclusion 

Project RETAIN suffered from man- 
agement difficulties. The CBE is 
large enough that these problems 
apparently were undetected (just as 
the Project itself operated without 
the knowledge of the General Super- 
intendent until her communications 
with OJJDP) despite each proj ect 
modification being cleared by a DFS, 
an administrator pi aced rather high 
in the system. Although the manag- 
ers had all worked with^ this DFS, a 
formal hierarchy in decision making 
appeared to be the rule, perhaps 
accounting for the minimal awareness 
of Project RETAIN at higher adminis- 
trative levels. The assignment of a 



Government Funded Programs monitor 
in the second year did not help to 
resolve the management difficulties. 

Teachers in the school s tried 
val iantly to implement the proj ect 
to the best of their abil ity , but 
lacked guidance and training in the 
many skills needed. Students at all 
sites seemed to be attached to the 
RETAIN teachers and grateful for 
their help, but management practices 
resul ted in nine different programs 
evolving (one in edch of the 
schooLs) rather than the development 
of a single cohesive program. 

The large Chicago Board of Educa- 
tion bureaucracy is a highly politi- 
cal organization. Managers were 
sensitive to the pol itics of the 
system. They appeared less sensi- 
tive to the demands of developing a 
strong set of interventions, and had 
difficulty in working with the pro- 
ject ''s complex forcefield to develop 
the project. The very nature of the 
project meant that the system had to 
be modified at several levels to 
accommodate it. Project activities 
and organization required that new 
lines of administration be developed 
to assist in the direct contact bet- 
ween RETAIN teachers and managers. 
Without these arrangements, the 
question of who was in charge — prin- 
cipal or project managers-was left 
wide open, and the project took on a 
different character at each school . 
The project dropped entire compo- 
nents at some schools, and added 
additional activities at others, in 
large part in accommodation to the 
principals of the several schools. 
New pol icies were needed for the 
impl ementa t ion of many pi anned 
activities such as campus release 
time for home visits. 

It may have been helpful if pro- 
j ect managers had been empowered to 
work . with pr incipal s to pi an and 
impl ement exper iment al pol icies and 
procedures that would have affected 
their schools. Managers did not 
perceive they had this power. 
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Figure 1: Aspects of Chicago Board of Education 
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Table 1 



Numbers of Treatment and Non-Equivalent 
Control Students, 1981-82 School Year, 
in Each Project School — Project RETAIN 



Treatment Status 



Non-Equivalent 
School Treatment Control 



1240 


15 


18 


1340 


14 


15 


1430 


18 


25 


2300 


16 


20 


4440 


15 


16 


5090 


18 


17 


5750 


18 


20 


5880 


16 


17 


6180 


15 


15 
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Table 2 



Average Proportion of Missing Pre-Treatment Data 
from 1980-81 School Records in Each Project 
School, 1981-82 School Year, Project RETAIN 



Treatment Status 



School Treatment Control . Overall 



1240 


.33 


... -1* 


.23 


1340 


.30 


.33 


.32 


1430 


.27 


.72 


.53 


2300 


.08 


.02 


.05 


4440 


.05 


.20 


~ .13 


5090 


.17 


.26 


.21 


5750 


.06 


.12 


.09 


5880 


.58 


.51 


.54 


6180 


.02 


.05 


.03 


Project as 
a whole 






.24 
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Table 3 



Average Proportion of Missing Post-Treatment Data 
from School Records in Each Project 
School, 1981-82 School Year, Project RETAIN 





Treatment 


Status 




School 


Treatment 


Control 


Overall 


1240 


.32 


.67 


.51 


1340 


.65 


.63 


.64 


1430 


.60 


.82 


.73 


2300 


.16 


.22 


.20 


4440 


.11 


.36 


.24 


5090 


.24 


.10 


.18 


5750 


.33 


.39 


.36 


5880 


.88 


.88 


.88 


6180 


.10 


.19 


.14 


Project as 
a whole 






.44 
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Table 4 

Average Proportion of Missing .Student Questionnaire 
Outcome Data for Each Project 
School, 1981-82 School Year, Project RETAIN 



Treatment Status 



School Treatment Control Overall 



1240 


.63 


.79 


.72 


1340 


ft '5' 

.93 


1 .00 


.9/ 


1430 


.35 


.77 


.59 


2300 


.14 


.24 


.20 


4440 


.21 


.48 


.35 


5090 


.19 


.16 


.18 


5750 


.44 


.38 


.40 


5880 


.62 


.46 


.5~3 


6180 


.14 


.36 


.25 


Project as 
a whole 






.46 
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Table 5 



Comparison of Background Characteristics 
of Project RETAIN Treatment and Control Students 
(Entire Sample) During 1981-82 School Year 



Group 



Non-equivalent 
Treatment Controls 



Variable 


M 


SD 


N 


M 


SD 


N 


Sex (coded 1-2) 


1.36 


.48 . 


107 


1.37 


.49 


105 


4th Quarter English 
Grade~1980-81 


.58 


.73 


117 


.59 


.68 


130 


1980-81 ITBS Reading Score^ 


6.33 


6.90 


96 


5.41 


5.75 


103 


1980-81 ITBS Math Score^ 


6.46 


6.56 


95 


5.36 


6.68 


102 


Credits Earned Through 
June 1981^ 


3.68 


3.27 


28 


2.54 


2.82 


48 


Total Days Absent During 
1980-81 School Year 


21.83 


18.20 


101 


23.85 


22.97 


110 


Total Days Lost Due to 
Suspension : 1980-81 


.18 


' .52 


102 


.27 


.74 


97 


Age (from 1982 
Student Survey) 


13.44 


1.66 


95 


13.14 


1.70 


86 


Grade Level (from 1982 
Student Survey) 


1.78 


1.45 


95 


. 1.54 


1.33 


83 


Parental Education (from 
1982 Student Survey) 


2.08 


1.20 


72 


2.16 


1.15 


58 


^Variable is available only for students who 
j^during the 1980-81 school year. 
Variable is available only for students who 


were 
were 


in elementary school 
in high school durinj 



the 1980-81 school year. 
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Table 6 



Background Differences 
of Project RETAIN Treatment and Non-Equivalent Control Students 
During 1981-82 School Year, by School 



Group 





Treatment 




Non-equivalent 
Controls 




Variable 


M 


SD 


N 


M 


SD 


N 


Prob. 








School 


1240 








Total Days Lost Due to 
Suspension: 1980-81 


.25 


.46 


8 


.93 


1.07 


14 


.05 








School 


1340 








Total Days Absent During 
1980-81 School Year 


39.15 


19.18 


12 


55.04 


20.64 


13 


.06 








School 


1430 


_ — — 


— 


-. _ 


1980-81 ITBS Math Score 


12.08 


13.23 


7 


.63 


1.51 


12 


.06 


Credits Earned Through 
June 1981 


5.39 


3.86 


11 


1.58 


3.05 


18 


.01 








School 


5090 








Age (from 1982 
Student Survey) 


13.31 


1.14 


16 


12.33 


1.84 


15 


.09 


Grade Level (from 1982 
Student Survey) 


1.53 


.64 


15 


.78 


.80 


14 


.01 








School 


5750 








Total Days Absent During 
1980-81 School Year 


16.28 


18.73 


16 


6.42 


10.62 


18 


.08 








School 


5880 








Age (from 1982 
Student Survey) 


12.28 


1.11 


7 


13.30 


1.06 


10 


.08 


Note. These comparisons 


suffer from 


problems 


of missing data. 
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Table 7 



Comparison of 1981-82 School xear Outcome Variables from 
School Records for RETAIN and Control Students 



Group 





Treatment 




Non- 


•equivalent 
Controls 


Outcome Variable 


M 


SD 


N 


M 


SD 


N 


Ratio of Dav s 
Absent to Days 
Enrolled: 1981-82 


.10 


.14 


101 


.10 


.13 


80 


Ratio of Days 

Excused Absence to Davs 

Enrolled: 1981-82^ 


.03 


,04 • 


68 


.03 


.04 


65 


Unexcused Absence to Days 
Enrolled: 1981-82 




OS 


61 


.02 




62 


Ratio of Days Lost 
Due to Administrative 

T? lam ^^7 sil t*n Dave 

Enrolled: 1981-82 


.00 


.01 


78 


.01 


.02 


73 


T?fit*in nf Tittip*; Sii<?npndpd 

to Days Enrol led : 1981-82 


.00 


.01 


83 


.00 


.01 


59 


Average English Grade 

\J. *-r 9L.CIXC/B 


.89 


.66 


95 


.86 


.69 


84 


Average Math Grade 
(1-4 scale): 1981-82^ 


.88 


.67 


87 


.94 


.60 


77 


Average Science Grade 
(1-4 scale): 1981-82^ 


.96 


.70 


64 


.91 


.60 


64 


Average Social Studies 
Grade (^-4 scale): 
1981-82^ 


.93 


.64 


70 


.89 


.57 


67 


Dropout 


.15** 


.36 


145 


.35 


.48 


163 



Note. The statistically significant outcome for dropout favoring the 
treatment group dissappears when appropriate statistical controls are 
applied. 



Variable is applicable only for project elementary schools. 
^Average of the four quarterly grades in the particular course. 
** p<.01 
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Table 8 



Comparisons of 1981-82 School Year Outcome Variables from 
Student Survey for RETAIN Treatment and Control Students 

(Entire Sample) 



Group 



Non-equivalent 
Treatment Control s 



Var iabl e 


M 


SD 


N 


M 


SD 


N 




53* 


31 

• ^ X 


87 


62 


96 


7 8 
/ o 


A t* t* p ViTP pnt* tTi Pr»T*OTit*c 


• •+ 


• ^ u 


Q9 


58 
• ^ o 


98 
• ^ o 


85 


Np<>flf*ivp Pppt" Tnflnonr»o 


• u ± 




oo 


3 1 
• J 1 


' 9 1 
• ^ 1 


8A 
CD 


At* t* arhmpn tn ^rhnnl 






87 


5A 


3 n 


81 
Ol 






94 


83 




98 
• ^ o 


IL 
1 


Tnt*OT*T^OT*coTia1 r'rtTPiT^o t* on /*v 
JLllLCLpcLoUllctX V^UUipc L CU 




• ^ J 


8A 


7 7 


9*^ 


7 A 






1 7 


7 S 


65 


1 Q 

• 1 7 


SQ 

J 7 


ocix ivcpuL Led i/ci muuciiuy 


• 1 o 


9'^ 


01 


1 Q 




7 n 
/ U 


Qpol p 














fipl "F^R pnm* f* pH 'H'tiio TTco 


28 


• J u 


83 


9 1 
• ^ 1 


33 


7 9 
/ z 


^PT*TOIlC Tlol ITInilOTl/'V 


• 1 J 


9 1 


81 

Ol 


• 1 J 


9 5 


7 1 
/ 1 


Piin 1 cViTTipnt* TnHov 


• 


9Q 


86 


9 5 


9 5 

• Z J 


77 


RpTjfli^Hc TnHov 

IxCWdl. UO JLllUCiv 


9 8 
. • ^ o 


1 

• J 1 


8S 


97 


97 
• z / 


7 8 
/ o 




1 Q 


9S 


86 


1 7 
• 1 / 


9 1 
• ^ 1 


78 
/ o 


School Effort 


.49 


• ^ ^ 


95 


47 


3 1 

• ^ X 


83 
o^ 


PT*flPt*iPfll TTnritjlpHop 
^ X d I. X dx iviiuwxcugc 


1 46* 




83 


1 33 


A5 


75 


Tnt*OT*nQl — T7vt*o^nQl r'ont*^o1 
XllLCLllclX uALcLUdl V^OULlOI 




9 9 


83 


S3 


93 
• Z J 


7 S 


Al lonat*! on 




96 


81 

Ol 


A7 


9 9 


7 S 


^p1 f — RpnoTt*pH (Ifa/^aG 


±•71 


± •\JO 


Q3 


1 Q3 
1 • 7 J 


1 • Uf 


8A 


Reading Ability, ^elf Rating 


1.11 


.90 


89 


1.14 


.81 


77 


Days of School Cut in 














4 Weeks Preceding Survey 


.56 


1.09 


95 


.38 


.72 


85 


How Often Cut One or 














More Classes 


.98 


1.41 


95 


.74 


1.23 


86 


Educational Expectation 


2.89 


1.85 


95 


2.93 


1.90 


85 


Did You Work for Pay Last 














Week? 


.65** 


.48 


95 


.36 


.48 


86 


Regular Part-Time or 














Full-Time Job? 


.37 


.58 


95 


.28 


.57 


86 


Suspended from School During 














Term 


.43 


.50 


86 


.33 


.47 


78 


School Nonattendance Index 


.72 


.84 


94 


.62 


.77 


85 


Rebellious Autonomy 


.68 


.33 


83 


.63 


.30 


73 


Involvement 


.27 


.21 


87 


.23 


.20 


81 



Note . Two of the statistically^^ significant differences in these raw 
comparisons (i.e. Parental Emphasis on Education, Practical Knowledge) 
are not significant when appropriate statistical controls are applied. 

* p<.05 

**p<.01 
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The Nilwood ''^ative Project: Second Interim Report 



M. S. Cook 



Abstract 

The Mil wood Alterna ^ Project is one of several projects funded 
by the Office of Juvenile. Justice and Delinquency Prevention (OJJDP) 
as part its Program in Delinquency Prevention through Althernative 
Education. The Mil wood Project operates in Mil wood Junior High 
School in Kalamazoo, Michigan. Project components during 1981-82 
included an in-school suspension room, a home-school liaison, an 
attendance clerk, a Sk lis Lab class for low-achieving students, a 
school -within-a-school for eighth grade students (the Milwood Alter- 
native Program) , a student council , a pr oj ect advisory , and an 
intramural/activities program. 

School climate comparisons with South Junior High School Kthe 
control junior high ) indicate that between the 1981 and 1982 school 
years school climate improved slightly at Milwood, and declined sub- 
stantially at South. Milwood is maintaining a positive school cli- 
mate in the face of financial cutbacks in Kalamazoo programs. Basic 
skills scores improved significantly at Milwood, while remaining 
stable at South. We cannot conclusively attribute these positive 
effects to the Milwood Alternative Project. 

Chronic non-attendance — skipping several consecutive days of 
school — is also down at Milwood as compared to South. The in-school 
suspension room reduced the number .of exit suspensions, but does not 
appear to have affected the overall number of suspensions. We are 
unable as yet, to judge the effectiveness of the other project 
interventions. For 1982-83 an Intensive Study Program has replaced 
the Skills Lab, and a reading diagnosis and tutoring program has 
been added. 



Major Program Changes in 1981-82 

The Milwood Alternative Project 
continued implementing the maj or 
program interventions described in 
the first interim report (Carlton 
and Cook, 1982). Briefly, these 
interventions included (a) the Mil- 
wood Alternative Program, a school- 
within-a-school for sixty eighth 



This report covers roughly the 
period of September 1981 to June 
1982. 



grade students, (b) the Skills Lab, 
a basic skills class, (c) a student 
council, (d) a student project advi- 
sory — a student board with input 
into project activities, (e) a pro- 
ject adv isory ut il iz ing communi ty 
members , (f ) an attendance monitor 
and home-school liaison, and (g) an 
in-school suspension center. The 
only modification of note was in the 
teaching staff for the Skills Lab 
class. Because of teacher layoffs 
and teacher union issues related to 
seniority, the original teacher for 
the Skills Lab (around whom the 
class was designed), was replaced 



227 



ERiQ 



ioo 



Mil wood Project 



first with substitute teachersi then 
with a school counselor appointed by 
central administration who had never 
taught in a classroom- Finally, in 
January the position was filled by a 
teacher selected by Milwood staff 
froia a pool available for seniority- 
based layoff recall. This person 
did not have specific expertise in 
working with low-skills children, 
and was not involved in program or 
proj ect planning* 

Maior Forcefield Changes in 1981-82 

Four major f orcef iel d changes 
occurred between the 80-81 and 81-82 
school year: a change in the Mil- 
wood assistant principal ; the com- 
mitment of the Milwood principal and 
new assistant principal to project 
success; the development of strong 
support for the project within the 
community; and budget cuts forcing 
teacher layoffs. 

Partly due to project efforts, 
the 80-81 assistant principal, who 
was responsible for reconunending the 
high number of out-of-school suspen- 
sions experienced at Milwood during 
the 80-81 school year, was replaced 
by an assistant principal who is 
more accepting of alternative forms 
of discipline (i.e., referral to 
in-house rather than exit suspen- 
sion). Also, the building principal 
has become an advocate for the pro- 
ject, both within the school and in 
the community at large. Although 
not involved in the early stages of 
project development, Milwood's prin- 
cipal has become actively involved 
in project activities, and now 
serves as an enthusiastic proj ect 
spokesperson before parent groups, 
central administration staff, and 
the media. 

Community support for the project 
increased sharply during the 81-82 
year. For example. The Citizens' 
Coalition on Discipline (a task 



force set up to evaluate the 
Kalamazoo Public Schools' discipline 
policies) has praised the Milwood 
project's in-school disciplinary 
policies, and recommended that the 
policy be implemented system-wide. 
Favorable reviews of the proj ect 
appeared in the local newspaper, on 
the T.V. nex^s, and representives of 
the project were invited to discuss 
their work on a local television 
talk show. The Milwood project has 
also received favorable press 
through a UPI news release. 

A negative force acting on the 
project was the financial situation 
of the state of Michigan and the 
city of Kalamazoo- Although the 
recession has not had as much of an 
impact on Kalamazoo as on the rest 
of the state, budget cuts did result 
in widespread teacher reductions 
from 1981 to 1982. Besides gener- 
ally reducing personnel and 
resources from th~e schoDl (pr'bbably 
negatively affecting school climate) 
the cuts had a particularly negative 
effect on the project's Skills Lab 
class. 

The teacher who had been instru- 
mental in designing the class, and 
who was to teach it was bumped down 
to the elementary school level due 
to her lack of seniority. The posi- 
tion was covered by several diffe- 
rent teachers throughout the year , 
and was never filled by a person 
experienced or trained in teaching a 
basic skills class. 

In short , some major forcefield 
changes impinging upon the proj ect 
were positive : Extensive support 
for the project has developed in the 
school itself, in the school system, 
and in the community. One force- 
field change x^as negative: teacher 
layoffs based on seniority rather 
than training or ability. 



228 



ERIC 



101 



Milwood Project 



Evaluation Results 

School Climate 

Because the Milwood Alternative 
Education Proj ect is primarily tar- 
geted at school-wide climate change, 
school level comparisons are of 
major interest. 

Milwood and South in 1980-81 . 
The 1980-81 school year was predomi- 
nantly a planning year for the Mil- 
wood Alternative Proj ect (see the 
first interim report). Most project 
components were not implemented 
until the 1982 year. Thus, the 1981 
questionnaire resul,ts may be con- 
strued as pre-treatment data. The 
findings in Table 1 support the 
claims of project staff that Milwood 
suffered from a poorer school cli- 
mate than South at the initiation of 
project activities. There were six 
significant differences on the 1981 
student psycho-social scales. Mil- 
wood students , as compared to South 
students , had lower sel f -esteem and 
higher rebellious autonomy. Milwood 
students were also suspended more 
often, attended school less, and 
reported more del inquency and drug 
use. 

It should be no ted that Milwood 
had a lower percentage of minority 
students than did South in 1981 (31% 
vs 45%, p = .005), and that race was 
correlated with several of the ques- 
tionnaire seal es. Those that were 
significant at the .01 level or bet- 
ter may be found in Table 2. Being 
white is associated with greater 
bel ief in the rules , but less prac- 
tical knowledge, and less involve- 
ment in conventional activities. 
Minorities are suspended more often, 
receive lower grades, and their 
parents have completed fewer years 
of education. Whites report more 
drug usage, r 

In order to control the effects 



of the differential racial composi- 
tion of the two schools, an analysis 
of covar iance was performed. 
Because parental educational level 
also tended to differ between Mil- 
wood and South (see Tabl el) paren- 
tal education was also introduced as 
a covariate. Table 3 presents the 
comparisons that were significant 
after controlling for race and 
parental education. 

Striking differences remain bet- 
ween the two schools after correct- 
ing f pr the two demographic varia- 
bles, race and parental education. 
The adjusted means show that Milwood 
students, as compared to South stu- 
dents, reported higher rebellious 
autonomy in 1981. More importantly, 
perhaps , Milwood students were sus- 
pended much more often than South 
students , reported more del inquency 
and drug use , and skipped school 
more often. 

The foregoing analyses indicate 
that the climate at Milwood was" less 
favorable than that at South at the 
close of the 1981 school year. Mil- 
wood students were misbehaving more, 
getting into more trouble at school, 
and cutting school more often. The 
question becomes what happened as 
the project began implementation in 
earnest? 

Milwood and South in 1981-82 , 
Table 4 documents the resul ts from 
analyses of variance carried out on 
the 1981-82 SAES questionnaire sca- 
les. Major differences can be noted 
between these results, and those 
found in Table 1. Table 4 indicates 
that by the end of 1982 Milwood stu- 
dents, as compared to South stu- 
dents, reported greater attachment 
to school, found school less punish- 
ing, vere less al ienated, and were 
victimized less. On the other hand, 
Milwood students said they were now 
less involved in school activities 



229 



102 



Mil wood Project 



than South students, and said that 
they were poorer readers. Mil wood 
students also reported greater prac- 
tical knowledge. As contrasted with 
1981, there were no differences on 
the SAES questionnaire drug use, 
attendance, or suspension measures, 
and the trend was for Milwood stu~ 
der.ts to report less delinquency 
overall , and less serious del in- 
quency . Student characteristic inea- 
sures show that Milwood students 
rated themselves as having less 
reading ability than students from 
South. 

An analysis of - covariance cont- 
rolling for race and sex produced 
essentially the same analyses, and 
will not be reported here. 

These results indicate that the 
climate at Milwood had improved 
relative to South between the spring 
of 1981 and the spring of 1982. Not 
only did Milwood students like 
school more than the South students, 
and report themselves to be less 
alienated, but they no longer dif- 
fered from ^ the South students in 
their self-reported attendance or 
drug use. Importantly, in 1982, 
Milwood students reported themselves 
to have been victimized less than 
South students and scored lower on a 
measure of al ienation. There are 
also trends toward Milwood students 
repor ting less del inquency overal 1 
and less serious delinquency. 

But the positive changes at Mill- 
wood do not tell all of the story. 
Student involvement at Milwood 
remains low (as compared to South), 
and in 1982, large differences in 
self-reported reading ability exist, 
which appear to be the result of the 
joint effects of improvement at 
South, and a slide in self-reported 
reading ability at Milwood (see 
Table 1). The 1982 differerce in 
repor ted reading abil i ty shoul d be 
interpreted cautiously since analy- 



sis of the Metropolitan Achievement 
Test of basic skills (MAT) indicates 
that Milwood students were indeed 
poorer readers than South students 
in 1981, but this was not the case 
in 1982 (see details, below). 

The finding of less involvement 
on the part of Milwood students 
impl ies that the project has been 
unsuccessful to date in signifi- 
cantly increasing Milwood students' 
involvement in school extra--curricu- 
lar activities. Both project staff 
and the school administration attri- 
bute great importance to extra-cur- 
ricular activities in the promotion 
of school attachment, thus the find- 
ing of low involvement in extra-cur-- 
ricular activities is an important 
outcome for further program develop- 
ment efforts. Likewise , project 
staff theorize that basic skills 
prof iciency is important for 1 if e 
success (and reducing the risk of 
delinquency) , ^ so' the Tiecrease in 
reported reading abil ity scores 
found ^at Milwood may also suggest 
programmatic change. 

CI imate over t ime . The next set 
of analyses was conducted to examine 
' changes in the SAES scores from 1981 
to 1982. The first set of analyses 
determined that in 1981, Milwood had 
a. less favorable school climate than 
South. This set of analyses are an 
attempt to ascertain whether the 
generally more favorable climate at 
Milwood in 1982 as compared to South 
was due to Milwood developing a more 
positive school cl imate , South a 
more negative, or somie combination 
of the two. 

In the first attempt to investi-- 
gate changes in school cl imate from 
the 1980-81 to the 1981-82 school 
year, change scores were calculated 
for those students for whom there 
were completed questionnaires for 
both years. These, of course, would 
be for the most part students that 
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were seventh graders in 1980-81 and 
eighth graders in 1981-82. 

Table 5 gives the results for 
this series of analyses of variance 
on the available SAES scales (inter- 
nal Control was not scored for 
1980-81). Tlie only change score 
that differed significantly for Mil- 
wood and South was in self-reported 
reading abil ity : Milwood showed a 
decrease of .13 in sel f-repor ted 
reading ability, while South showed 
an increase of .34, i.e.. So y^h stu- 
dents reported themselves to be 
somewhat better readers, while Mil- 
wood students reported themselves to 
be slightly poorer readers. This 
result is consistent with earlier 
analyses. 

There is also a trend toward Mil- 
wood students becoming more alien- 
ated over the 1981-82 year (p=.07). 
No other comparisons approached sig- 
nificance. 

This analysis is difficult to 
interpret due. to the relatively 
small numbers of individuals on 
which we have data for both years; 
N's range from 52-84 for the change 
scores as compared to 191-676 for 
the simple between-groups , 1982 com- 
parisons. The change scores do not 
include any student who had spent 
only one year at either school, and 
largely exclude seventh graders (and 
therefore the sampl es differ in age 
also) . 



The effect of the more select 
(and older) sample size can be seen 
in the al ienation measure. While 
the change scores indicate a big 
jump at Milwood and a very small 
decrease at South, means for the 
entire yearly sample at each school 
(Tables 1 and 4) show that, overall, 
alienation was up somewhat at both 
schools. 

A further consideration is that 



the Milwood students included in the 
change score sample had only experi- 
enced one year in which project 
interventions were fully opera- 
tional. Project staff hypothesized 
that the eighth grade class of 
1981-82 was particularly resistant 
to influence, largely because of 
their early (seventh grade) negative 
exper ience s in the school env iron- 
ment; as a school Milwood was less 
harmonious in 1980-81 than in 
1981-82 • Students exposed to pro- 
ject activities in 1981-82 may have 
alrea4y "opted out" of school. 

Summary of school climate 

results . The pattern of results 
from the two sets of data — the bet- 
ween school analyses, and the 
changes from 1981 to 1982 — suggest 
three conclusions concerning school 
climate at Milwood and South: 

1. Milwood exhibited a slight 
improvement in school climate from 
.1981 to 1982 . The school climate at 
j_outh showed negative changes during 
t_his saiTQ period . 

Comparing Tables 1 and 4 it is 
apparent that from 1981 to 1982 Mil- 
wood and South reversed their ove- 
rall standing on school climate. 
Whereas South had a better school 
climate in 1981 than Milwood, the 
reverse is true in 1982. The mea- 
sures of attachment to school, 
al ienation, rebel 1 ious autonomy , 
practical knowledge , sel f-repor ted 
school non-attendance , suspension, 
delinquency, drug use, and victimi- 
zation all support this interpreta- 
tion. 

2. Measures of Involvement in 
conventional activities and self-re- 
ported reading ability favor South 
over Milwood . These results are 
puzzling. Involvement in school-re- 
lated activities was a major Milwood 
project focus in 1982. Students at 
Milwood apparently did not feel its 
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impact, or not enough students were 
involved in project activities late 
in the school year (at the time of 
the SAES questionnaire administra- 
tion) to differentially affect the 
Involvement measure. Tlie self-re- 
ported reading ability measure is 
highly suspect in these data 
because, although Milwood^s self-re- 
ported reading scores declined bet- 
ween 1981 and 1982, and South's 
improved, test scores shov precisely 
the opposite pattern. 

3 . It is impossible to unequivo - 
cally attribute the pattern of 
school climate results ( relative 
improvement at Milwood ^ and decline 
at South ) to Milwood project activi - 
ties . 

It is reasonable to argue that 
the less favorable climate at South 
in 1982 as compared to 1981 is a 
result of restricted resources and 
cuts in programs resulting from the 
recession. These cuts translated 
mainly into staffing cuts , and cuts 
in extracurricular programs. The 
slight positive climate changes wit- 
nessed at Milwood in the past year 
are impressive because they have 
come during a time of generally 
declining resources for education in 
the area. In other words, this 
argument assumes that without the 
al terna tive pr oj ect efforts, the 
climate at Milwood would have deter- 
iorated as it did at South. 

An al ternative explanation was 
offered for the decline in school 
climate at South by its principal . 
She feels that the negative climate 
effects in 1982 were due to a parti- 
cularly "bad" eighth grade cl ass ; 
that particular cohort at South 
caused the problems. This would not 
explain why the effects did not show 
up until these students^ eighth 
grade year, since they had been at 
South during the previous school 
year. 



A third explanation for the 
effects might 1 ie in regression 
artifacts . Milwood was chosen for 
project activities because it was 
judged to have the poorest school 
climate of the three junior high 
s chool s in Kal amazoo . If Milwood 
was below the mean in school climate 
at the start of the project it would 
tend to move toward the average cli- 
mate of the area schools, and the 
results over time would be expected 
to look much as they do. At this 
juncture, we can only S'ay that Mil- 
wood is sh ow ing pr om i s ing impr ov e- 
ments in school climate, but we must 
wait for the third year results for 
,^a more definitive assessment of ove- 
rall school climate change. 

Metropolitan Achievement Test Scores 

A further way of examining organ- 
izational changes is by analyzing 
the performance of the students of 
the two schools on the >Ietrop~ol it'an 
Achievement Test (MAT) , a standard- 
ized basic skills test. The MAT 
provides scores on reading, use of 
language , basic mathematics , and an 
overall composite (total) score. 

Table 6 presents the spring 1981 
raw and percentile scores for the 
three subtests and the composite 
s core for Milwood and South . The 
mean scores presented are adjusted 
for grade , sex, race , and age. On 
all three subtests and the composite 
scores, the South students performed 
significantly better than the Mil- 
wood students. The differences are 
not only statistically significant , 
but are of a fairly substantial sub - 
stantive size al so. The composite 
percentiles indicate that Milwood 
students (after controll ing for 
background differences ) scored at a 
level that was eight percentile 
points below that of South students. 

The following table. Table 7, 
gives the results for the analyses 
of the spring, 1982 MAT scores. 
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These results are dramatically 
different from those of Table 6. 
Milwood students no longer differed 
from South students in Reading, Lan- 
guage, or on the overall composite 
score, and actual ly showed signifi- 
cantly better performance on the 
mathematics test . Tables 6 and 7 
indicate that the 1982 effects are 
the result of virtually no change in 
performance at South between 1981 
and 1982 , but of large increases at 
Milwood — eleven percentile points on 
the composite score. 

In order to examine the hypothe- 
sis that changes between the years 
are due to cohort differences, an 
analysis of variance was conducted 
on both the 1981 and 1982 MAT scores 
that included school, grade, sex, 
and race as factors, and controlled 
for the age of the student. This 
analysis indicated that, in general, 
whites perform better than blacks 
and other minorities, and girls per- 
form slightly better than boys. 
These effects were similar for both 
schools, and for both years. Table 
8 gives the adjusted means for the 
race and sex main effects for the 
overall composi te score. The dif- 
ference between the overall grade 
level s within a school are largely 
uninterpretabl e because both the 
seventh and eighth graders take the 
seventh grade version of the MATs, 
and the eighth graders would be 
expected to score more highly on the 
tests. 

More importantly , the only con- 
sistent interaction effect was bet- 
ween school and grade in 1981. A 
significant interaction effect indi- 
cates that the results for one vari- 
abl e (in this case , the student's, 
grade in school) depend upon another 
variable (here, the school in which 
the student was enrolled) . Tabl e 9 
shows the means for this interac- 
tion. These means, adjusted for 
sex, race, and age, show that the 



poorer performance of Milwood versus 
South in 1981 was entirely due to 
differences in performance of stu- 
dents in the two seventh grades. 
The two groups of 1981 eighth grad- 
ers scored very similarly on the 
tests* 

Table 10 gives the same results 
for the 1982 scores, and Table 11 
lists the average percentile change 
from 1981 to 1982 for the school by 
grade groups . As Tabl e 11 docu- 
ments, both the 1982 Milwood seventh 
and eighth grade cohorts scored 
higher than their 1981 counterparts, 
with the seventh grade cohort show- 
ing the biggest increase (and 
accounting for the disappearance of 
the interaction effect found in the 
1981 data). The 1982 eighth grade 
students (who are essentially the 
same students as the 1981 seventh 
graders, and who do not constitute a 
different cohort) gain eight percen- 
tile points on the composite score. 
The overall change from 1981 to 1982 
on the composite index was +12 per- 
centile points for Milwood students, 
and +1 point for South students. As 
seen in Table 11, the interaction of 
school by year (controlling for sex, 
race, and age) on the composite per- 
centile score is signif icant 
(p<.001). 

To summarize the MAT data. Mil- 
wood students (particularly seventh 
graders) performed much better on 
the 1982 than the 1981 MATs. South 
students showed 1 ittl e change bet- 
ween the two years. As was the case 
with the school climate findings 
derived from the SAES questionnaire, 
we cannot unambiguously attribute 
the positive gains at Milwood to 
project efforts. This is especially 
true given that there was not a 
great deal of attend \on given to 
basic skills education by the pro- 
ject during the 1981-82 year (see 
the analysis of the Skills Lab, 
below). Changes in administration 
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(or other, unknown changes) may 
account for Milwood^'s improvement in 
test scores. The third year results 
will be more definitive, but, at 
this point, the basic skills test 
outcomes are promising. 

Specific Project Components 

The next section of the report 
will discuss data pertinent to an 
evaluation of the individual treat- 
ment components of the Mil wood 
Alternative Education Project. 

The Milwood Alternative Program . 
No a_ priori control group was 
def ined for the MAP. Because of 
this the rest of the eighth grade 
class was used as the nonequivalent 
control group for the MAP. This is 
a conservative comparison. The MAP 
participants were invited to parti- 
cipate largely on the basis of hav- 
ing exhibited behavioral and aca- 
demic problems, so one would expect 
them to have poorer outcomes. 

The results for comparisons bet- 
ween Milwood Alternative Program 
(map) students and the rest of the 
Milwood eighth grade (all MAP parti- 
cipants are eighth graders) are 
shown in Table 12. Table 12 gives 
the results for analysis of variance 
comparisons between MAP participants 
and the rest of the eighth grade on 
the 1982 questionnaire scales- The 
results are mixed. Three variables 
central to the efforts of the MAP 
intervention. School Rewards, Inter- 
nal Control, and Interpersonal , Com- 
petency show significant differences 
that favor the MAP students. The 
Involvement in Conventional Activi- 
ties seal e also shows positive 
effects for the MAP* On the other 
hand, MAP students report lower 
self-esteem and attachment to 
parents than the rest of the eighth 
grade, and they report less parental 
emphasis on educa tion and greater 
negative peer influence. The MAP 
students also admit to more drug 



use, and to more serious delin- 
quency, and they are suspended more. 
There is also a tendency for them to 
report greater overall del inquency , 
and to hav e lower grade s . Final ly , 
the MAP students, as compared to the 
eighth grade at large, have much 
lower educational expectations. 

The difficulty with this analysis 
is , of course , the pr obabl e gross 
pretreatment differences between the 
MAP students and . the "control" 
group: the rest of 'the Milwood 
eighth, grade class. 

Table 12 suggests that pretreat- 
ment differences did exist: The MAP 
«j ... 
students are significantly older 

than the rest of the eighth grade 
(probably due to being held back in 
school), and they indicate uhat 
their parents have much less formal 
education than the parents of the 
typical Milwood eighth grader. The 
MAP stuclents are 'also" s"bmewhat ^mo're 
likely to be minority students than 
are the rest of the eighth grade. 

A second analysis was undertaken 
to attempt to control for those pre- 
treatment' differences identified 
above: age, 'parental education, and 
minority status. Table 13 shows the 
only significant differences that 
remain after controlling for the 
backgroud variables. MAP students 
report receiving more rewards at 
school than Milwood eighth graders, 
but they are suspended more of ten.* 
Thus, most of the differences that 
appeared in Table 12 can be attri- 
buted to the pre-existing differ- 
ences in the groups. 

In a sense, the lack of differ- 
ences between MAP students and 
eighth graders can be construed as a 
proj ect success. The MAP students 
were selected for the program on the 
basis of having behavioral and aca- 
demic problems . We would expect 
them to differ negatively from the 
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eighth graders had academic skills - 
that vere considered to be at too 
low a level for them to adequately 
function in the MAP program — where 
the instruction parallels that of 
the regular school curriculum—or 
were thought to be so disruptive 
that they would be a negative influ* 
ence on the other MAP students. 
Seventh graders in the program had 
academic problems or discipl inary 
difficulties, and did not qualify 
for special education or Title I 
programming. Thus , the Skil 1 s Lab 
students were a select group: they 
had weak academic skills, and were 
usually presenting disciplinary 
problems. 

Table 15 presents the results of 
AtlOVAs comparing the Skills Lab stu- 
dents first to the rest of the Mil- 
wood population (Skills Lab students 
could be in either the seventh or 
eighth grade), and second, to the 
MAP students. 

Looking at the differences bet- 
ween the SL students and the general 
Milwood student population, as one 
might expect given that the Skills 
Lsb students are selected because of 
having very poor basic skills, they 
differ from the other Milwood stu- 
dents on a host of measures. The 
Skills Lab students are less 
attached to school, find school more 
punishing, expend less effort in 
school, and report greater negative 
peer influence. They are also less 
attached to their parents and have 
lower self-esteem. Behaviorally , 
the Skills Lab students say that 
they are suspended more often, are 
more Jel inquent , use drugs more 
often, and receive lower grades. 
They are also older, expect to go 
less far in school, and (not sur- 
prisingly) , report being poorer 
readers. Finally, the Skills Lab 
students report that their parents 
are less educated than the typical 
Milwood parent. 



This analysis simply verifies 
what one would expect about a group 
of students that were selected for 
having large deficits in basic 
skil Is , and says 1 it tie about the 
success of the intervention in meet- 
ing the needs of those students. 
The differences between this group 
and the Milwood population as a 
whole were considered too gross to 
conduct an analysis controlling for 
background variables. 

Table 15 also gives* the results 
for signf icance tests for differ- 
ences between the SL and the MAP 
students. Again, the differences 
between the groups are extreme, and 
mirror the results from the compari- 
sons between the SL and the Mil wood 
students. (See Table 12 to compare 
MAP means with SL means.) Although 
the outcome differences were again 
large (and presumably, pretreatment 
differences were large as well), a 
series of " analyses' of~ covairiance 
were conducted on the scores of the 
SL and MAP students, controlling for 
parental education, reading ability, 
and age. Table 16 gives adjusted 
means and probability levels for 
those measures that remained signi- 
ficant. 

The psychosocial scales show that 
at the end of the 1982 school year, 
the SL students were less attached 
to school, expended less effort at 
school , and were less involved in 
school activities than were the MAP 
students (even while controlling for 
the background variables) . They 
also found school more punishing. 
Not surprisingly then, they expected 
to go less far in school. The SL 
participants also reported more 
negative peer influence. 

The behavioral self-report mea- 
sures indicate that the SL students 
engaged in more delinquency, drug 
use, and serious delinquency, and 
were suspended more in school. They 
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reported receiving lower grades than 
did the MAP students. 

Skills Lab students finished the 
1982 school year with much less 
fauorable outcomes than the Milwood 
Alternative Program students. 
Because we do not have the data to 
control adequately for pre treatment 
differences, the outcome differences 
between the SL and MP students are 
not very informative concerning the 
Skills Lab^s success. There are 
large outcome differences, but there 
were probably large input differ- 
ences between the two groups of stu- 
dents. 

It is apparent though , that the 
Skills Lab did not have large posi- 
tive effects on its students; the SL 
participants finished the year too 
poorly. In addition, project staff 
were disappointed in the program. 
Partly in response to the required 
OJJDP budget cuts, project staff 
decided to discontinue the interven- 
tion for the 1982-83 year. 

To what do we attribute the 
undistinquished performance of the 
Skills Lab? Interviews with project 
staff indicate one fundamental prob- 
lem: the teacher around whom the 
intervention was designed, and who 
contributed much toward the program 
in its planning stages in 1981, did 
not ultimately teach in the lab. 
School system budget cuts resulted 
in teacher layoffs and caused the 
original teacher to be transferred 
to the elementary level. Initially, 
her position was covered by substi- 
tute teachers , and later by a 
teacher selected by central adminis- 
tration, without consultation with 
the project. This person was a 
counselor and had never taught in 
the classroom, much less taught 
basic skills to academically handi- 
capped students. After project 
staff complained about the appoint- 
ment, and refused to pay the 



instructor's salary out of project 
funds, a search for a new teacher 
was undertaken. Tlie choices offered 
to the prciJject were still limited to 
those a'^'aiiable for recall based on 
seniority requirements. It was not 
until January that project staff 
identified a teacher that they hoped 
would be suitable. The final 

teacher was a former assistant high 
school football coach, who had been 
bumped "down" to the middle school 
level . He had no . experience in 
teaching low-skills students , and 
was nqt trained in special education 
(the teacher around whom the program 
had originally been designed held a 
masters degree in special educa- 
tion). Project staff understood, 
though , that he coul d be a patient 
teacher. Staff subsequently found 
that he was unexcited about teaching 
the basic ' skills class. He has 
since returned to football . In 
shor t , the Skil Is Lab had mul t ipl e 
teachers throughout the year , and 
never at any time, had a teacher 
experienced in or enthusiastic about 
teaching low-skills children. 

Home-school 1 laison . The home- 
school 1 iaison along with the atten- 
dance monitor (a secretarial -type 
position) are responsible for 
closely monitoring daily attendance, 
and intervening by letter , phone , 
and home visits — in a structured, 
sequential process — to insure that 
students are present at school. The 
attendance monitor collects daily 
absence reports, and prepares a 
daily list of students absent, which 
includes the number of consecutive 
days that the student has been 
absent . The home-school liaison 
then uses that information in making 
home contacts. (The home-school 
1 iaison is al so responsible for 
escorting students home that are 
given exit suspensions and whose 
parents cannot pick them up.) 
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The SAES self-reported attendance 
data shows no change from 1981 to 
1982 for either Milwood or South 
(see Tables 1 and 4). Individual 
level attendance data are not yet 
available at the writing of this 
report , but the aggregated school 
attendance data (as reported by the 
schools to the school district) can 
be examined. From 1981 to 1982 the 
average daily absence rate nearly 
doubled at South from 11% to 21%. 
During this same period, the absence 
rate decreased at Milwood from 13% 
to 11%. This pattern of effects 
resembles that found for the SAES 
school climate measures: little 
( though positive) change at Milwood 
accompanying a a large negative 
change at South. 

The home-school liaison spends a 
great deal of his time working with 
chronic non-attenders , students that 
are very often absent from school, 
usually for several consecutive days 
at a time. The records of Milwood 
and South were searched to identify 
students who missed 30% or more of 
the days for which they were 
enrolled to see if there was a 
change in the number of such stu- 
dents. At South during 1981 74 
children (11% of the average atten- 
dance period enrollment) fell in the 
non-attender category. In 1982 
there were 73 (11% of the average 
a ttendance period enrol Iment ) 
chronic non-attenders. In short, 
there was no change in the number of 
truant students at South . At Mill- 
wood there was a drop in the number 
of chronic non-attenders from 83 
(12% of the average enrollment) to 
57 (9% of the enrollment). This 25% 
drop in the proportion of truants at 
Milwood is statistically significant 
(z = 2.14, p < .05). 

These results suggest that the 
home-school liaison is successfully 
increasing attendance at school, 
particularly by increasing the 
attendance of students who would 



have been absent for several conse- 
cutive days. 

The In-school Suspension Center . 
As is the case with attendance, 
besides the SAES questionnaire mea- 
sures, only aggregate counts of sus- 
pension are currently available. 
The questionnaire data indicated 
little change in suspensions at Mil- 
wood, and an increase at South (see 
Tables 1 and 4) . The aggregated 
exit suspension information shows 
that as one would expfect with the 
installation of an in-school suspen- 
sion center, the number of exit 
(out-of-school ) suspensions decl ined 
dramatically at Milwood from 1981 to 
1982. In 1981 there were 510 exit 
suspensions levied, while in 1982, 
there were only 105. This repre- 
sents an 80% decrease in exit sus- 
pensions. At South, exit suspen- 
sions showed a moderate decrease 
from 195 to 170. \ 

It is impossible at this time to 
de t erm ine how much of th e dr ama tic 
decline in the exit suspension rate 
a t Mil woo d is due to a change in 
assistant principals or due to the 
development of the in-house suspen- 
sion room. Milwood^ s previous 
assistant principal handed out many 
more exit suspensions than the aver- 
age Kalamazoo discipl inarian; Mil- 
wood^s exit suspensionCrate for the 
1980-81 year was over twice the 
average rate of the other two Kala- 
mazoo junior high schools. The Mil- 
wood proj ect helped Idbby for a new 
assistant principal , and the new 
person does not resort as quickly to 
out-ofrschool suspension. 

At the same time, the 105 exit 
suspensions at Milwood in 1982 were 
considerably less than the 17 0 at 
both South and Hillside junior high 
schools . Consequently, the avail- 
ability of the in-school suspension 
room appears to supress the number 
of out-of-school suspensions. 
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The Student Council , Student 
Council representatives for each 
homeroom class were chosen by a lot- 
tery from the students in that home- 
room who nominated themselves for 
student council membership. This 
procedure produced a natural quasi- 
experiment. Table 17 gives the 
results for the MOVAs comparing the 
student council members to the ran- 
dom control s (those students not 
selected in the lottery). Perhaps 
the three most relevant SAES mea- 
sures in terms of the goal s of the 
program are attachment to school , 
school rewards, and involvement in 
school activities. Of these , 

attachment to school shows no effect 
of the program, but both school 
rewards and involvement in school 
activities show significant positive 
effects for the student council 
intervention. 

The picture is clouded when one 
examines the rest of the results of 
analysis; several negative outcomes 
are apparent. The student council 
students report more negative peer 
influence than the random controls. 
They also have lower self-esteem, 
and are less attached to their 
parents. They say that their 
parents put less emphasis on their 
education than the control students. 
Final ly , the student council stu- 
dents have lower educational expec- 
tations than do the controls. 

These results are puzzling, espe- 
cially those concerned with parental 
variables. There does not appear to 
be any theoretical reason why stu- 
dent council membership would affect 
these variables one way or another, 
much less negatively. Conceivably , 
since the student council ran some 
activities involving parents, the 
students "learned" that their 
parents did not care about their 
education, and therefore became less 
attached to their parents. This 
explanation seems a little far- 



fetched, and grants astonishing 
power to a rather weak intervention. 
Likewise, it is hard to see why stu- 
dent council membership would 
adversely affect oner's educational 
aspirations. 

This pattern of puzzling results 
tends to support the hypothesis 
that, despite an apparent randomiza- 
tion process, the student council 
students were a somewhat different 
population than the random controls. 
Although none of the f ol lowing dif- 
ferences are statistical ly signifi- 
cant , they consistently favor the 
control group over the student coun- 
,^ cil members. The student council 
members are more likely to be male 
than the controls (55% vs. 45%), 
i.e., girls were more likely to 
volunteer for the student council , 
but boys were more likely to be 
selected for it. The student coun- 
cil members^ parents tend to have 
had less education (p = .20) than 
the parents of the control students, 
and the student council students 
also report themselves to be 
si ightly poorer readers. In addi- 
tion, there is also a weak trend 
(p=«13) for the student council mem- 
bers to be suspended more often than 
the controls. Again, it is diffi- 
cult to imagine why being in the 
student council would cause one to 
get suspended. 

One possible explanation for the 
results is that student council mem- 
bers were chosen in a random lottery 
by homeroom ^ i.e. , from each pool of 
sel f-nominees from each home room 
class , a representative and an 
alternate were drawn. Because the 
MAP and Skills Lab classes consti- 
tuted homerooms, students with their 
qual ities may have been overrepre- 
sented on the student council . An 
analysis of variance was carried out 
which excluded those student council 
members and control s who were 
enrolled in either the MAP or Skills 
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Lab programs. Significant results 
from this analysis may be found in 
Table 18. 

As can be seen, thsise results are 
very similar to those reported in 
Table 17. The non-lIAP and non- 
Skills Lab student council members 
report lower parental emphasis on 
education, and less attachment to 
their parents. These student coun- 
cil members have lower self-esteem, 
and suffer from greater negative 
peer influence. They also work less 
hard in school. The student council 
members still report more school 
rewards (the effect is less signifi- 
cant because of the decrease in ji, 
but the absolute size of the differ- 
ence is the same). Oddly, the stu- 
dent council members that were not 
in the MAP and Skills Lab did not 
feel more involved in school activi- 
ties. To summarize this analysis, 
even removing the effects of a pos- 
sible overselection of students with 
behavior and academic problems, the 
student council apparently had nega- 
tive effects on the attitudes of *"he 
students that served on it. 

There are three possibilities to 
explain these anomalous results: 
(a) the randominzation process was 
not truly random, and some unidenti- 
fied selection bias was introduced 
(this possibility exists because 
each homeroom teacher did his or her 
own random selection from that 
room's pool of nominees, and we do 
not know if this procedure was car- 
ried out properly); (b) the lottery 
was random, and by chance, the 
groups were not equivalent; and (c) 
the lottery was random, the groups 
were ini tial ly equival ent , and the 
student council experience produced 
some negative attitudinal effects in 
the children. 

The Student Project Advisory . At 
the writing of this report, no 
information was available concerning 



the unique contributions or effects 
of the student project advisory. 

Summary of Second Year Evaluation 
Results 

1 . Milwood has shown some si ight 
increase in the quality of its 
school climate, while the climate at 
South (the control school) has 
declined. 

2. We cannot as yet conclusively 
attribute the differences in the 
school climate changes in the two 
schools to the Milwood Alternative 
Education Project, but it appears 

,^ probable that the project is favora- 
bly affecting the climate of Milwood 
Junior High School under conditions 
of declining resources available for 
Kal amazoo school s . 

3 . Milwood demonstrated signifi- 
cant and substantial improvement in 
MAT performance between 1981 ' and 
1982, as compared to South where 
performance remained the same across 
the two years. Both the seventh and 
eighth grades showed improvement at 
Milwood, with the seventh grade mak- 
ing somewhat stronger gains in 
scores. We cannot judge the impact 
of the Milwood Alternative Education 
Project on these basic skills gains. 

4. The MAP program is probably 
successful in creating a school 
environment that is more rewarding 
for academically marginal students. 
It is possible that the grouping of 
such children together may lead to 
some negative peer influence. 

5. There is no evidence that the 
Skil Is Lab was successful in 
increasing the positive school 
experiences of its students. 
Whether this was due to a lack in 
continuity and expertise in the 
teaching staff for the intervention, 
due to some problem inherent to the 
class, or due to pre-existing char- 
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acteristics of the students, we do 
not know. 

6 . Tlie home-school 1 iaison is 
probably successfully improving 
attendance at Milwood, particularly 
for students that are chronically 
absent. This intervention has been 
viewed so favorably by the school 
system that home-school 1 iaisons 
have been installed (at school sys- 
tem expense) in all Kalamazoo junior 
high schools. 

7. Most of the large decrease in 
exit suspensions at Milwood can be 
attributed to the change in assis- 
tant principals, but comparison with 
the other Kalamazoo junior high 
schools^ suggests that the availabil- 
ity of the in-school suspension room 
has also decreased the number of 
exit suspensions. 

8. The student council increased 
school rewards for its members, but 
did not help them feel any more 
involved in school. There is evi- 
dence that the student council may 
have had negative effects on the 
children's attitudes, perhaps 
through a negative peer influence 
mechanism. This possibility seems 
unlikely, so we await replication of 
these effects in the coming year's 
data. 



Changes Planned for 1982-83 

Changes in the Milwood Alterna- 
tive Education Project that are 
planned for 1982-83 fall into six 
general categories: (a) changes in 
the student composition, use of 
instructor time in the Milwood 
Alternative Program and time spent 
doing academic work; (b) elimination 
of the Skills Lab class and substi- 
tution of an individualized reading 
assessment and tutoring program; (c) 
addition of an intensive study pro- 
gram for selected fail ing students; 
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(d) greater and more systematic use 
of College student and adult volun- 
teers; (e) elimination of the atten- 
dance clerk and counsel or's secre- 
tary positions; and (f) increased 
emphasis on student involvement in 
school activities. 

Changes in _the_ Milwood Alternative 
Program 

The MAP was originally designed 
to include some "positive" influence 
students: students who- were having 
some dif f icul ty in their academic 
work, but who were not considered to 
be behavior problems. It was hoped 
that these students would seirve to 
set a positive example for the less 
wel 1-behaved students. In planning 
for the 1982-83 year, project staff 
decided that there had been too few 
of the positive influence students 
(only one or two in a class of 
twenty), and that they had not been 
"strong" leaders in the class. 
Therefore , their positive model ing 
was f el t to have been ineffective. 
For the coming year, a larger number 
of positive influence students have 
been included in each class, (al t- 
hough the numbers of positive stu- 
dents are still small) and students 
have been selected who are believed 
to be stronger peer leaders. It is 
hoped that the the greater number 
and influence of the positive stu- 
dents in the class will decrease 
some negative peer influence effects 
that project staff felt took place 
during the 1982-83 year. 

In addition to the inclusion of a 
greater number of positive influence 
students a different sel ection cri- 
terion was used for the majority of 
the MAP students. Students were 
sel ected who were judged to been 
attending school enough to be able 
to benefit from the treatment. In 
1981-82 , several students withdrew 
or were dropped from the program 
that never attended school regularly 
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from the beginning of the year; they 
never received the treatment. 
Chronic non-attenders were not 
selected for 1983. Also, it is the 
impression of project staff that 
they did not select students that 
had as severe discipline or academic 
problems as some of the students 
included last year. In short, the 
selection criterion were moved "up." 
This selection change was made 
because project staff felt that many 
of the students included in the pro- 
gram in its initial year were net 
responsive to treatment because they 
had, in effect, already dropped out 
of school, or were ' exhibiting disci- 
plinary problems too severe to be 
dealt with in a school -within-a- 
school . Staff bel ieve that these 
"worst" students were adversely 
affecting the rest of the students 
in the class. Such students were 
not invited to participate this 
year. 

Two changes have been made in the 
^lAP instructors' and counselors' 
time. Formerly during the first 
period of each day (the "group 
time"), one of the three school 
counselors assisted each of the 
three MAP homeroom instructors in 
carrying out the group activities. 
Building staff felt that this was an 
inefficient use of counselor time, 
and it did not provide any counselor 
"cover" for the rest of the Milwood 
population for that first period 
each day. Consequently, counselors 
rotate duties during the first group 
time period, with one of the counse- 
lors always remaining in the coun- 
seling department office. One of 
the MAP cl asses does not have a 
counselor, then, during group time, 
but the MAP staff feels that the 
instructors can carry out the activ- 
ities themselves since they now have 
a year of experience in facilitating 
the group time. 

A more fundamental program change 



of the instructors' time involves 
some extension of the MAP interven- 
tion into the afternoon, when the 
students attend regular classes. In 
1982, the MP teachers taught regu- 
lar classes in the afternoon, when 
the MP students were taking class 
with the rest of the Milwood stu- 
dents. For the upcoming year, one 
MP instructor's job has been res- 
tructured so that he is available in 
the afternoons to help MP students 
and their teachers in their main- 
stream classes. This '^support" for 
MP st,udenr.s in the afternoon, when 
they are not in the alternative 
environment , was deemed necessary 
for two reasons. First, the MP 
students still had difficulty in 
some of their non-program class'es, 
and it was felt that they and their 
teachers could benefit from some aid 
from a program teacher. Second, 
because the MP students take all of 
their morning classes together, 
there is a 'tendency "for ~them 'to' end 
up in the same classes in the after- 
noon due to a limited number of 
electives. In effect , some of the 
afternoon class teachers end up with 
a "MP" class when they do not have 
the training or desire to work with 
this group of students. It was 
thought desirable to have a MP 
instructor available to help teach- 
ers cope with this overabundance of 
"problem" students. 

Another important programmatic 
change involves the addition of an 
after school study hall for MP stu- 
dents. At weekly interval s , the 
students' progress on their written 
assignments is reviewed, and stu- 
dents who are performing poorly or 
have assignments outstanding are 
required to participate in an after 
school supervised study hall . Stu- 
dents and their parents are informed 
of tardy work on Friday, and if it 
is not completed, they are required 
to attend the study period on the 
f ol 1 ow ing Tue sday , Wedne sday , or 
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Thursday. Tlie MAP teachers rotate 
in supervising this special study 
hall. 

<> 

The last important change in the 
MP program involves the stucturing 
of the "group time." Group counsel- 
ing type activities are now held 
only three days a v;eek instead of 
five. The other two days are 
devoted to supervised study halls. 
Students work on their assignments 
or study for exams while receiving 
individual attention from their MAP 
instructor. 

To summarize the major changes in 
the MAP for the 1983 school year, 
they are of three kinds: (a) the 
program participants are expected to 
be better behaved, and to have bet- 
ter academic skills; (b) program 
support will be provided to the stu- 
dents outside of the regular morning 
program , and (c ) more time is being 
spent on academic pursuits, and less 
on group counseling. 

Changes in Basic Skills Instruction 

The Skills Lab class has been 
discontinued. It was not viewed by 
project and building staff as having 
been successful , and it was el imi- 
nated in reponse to the required 30% 
third year funding budget cut. 

Because the upgrading of basic 
skills is theoretically important to 
proj ect staff , an al ternative — less 
expensive — attempt to address the 
problem has been designed. A read- 
ing specialist has been hired who 
will be responsible for testing 
children identified by teachers and 
project staff as having reading 
problems. Academic success and MAT 
scores are the primary referral 
sources. The specialist is a gradu- 
ate student at Western Michigan 
University who holds a Masters 
degree in reading education. She 
will work at the school 2-3 days per 



week. Students will be tested to 
insure that they do have a reading 
problem and to diagnose the nature 
of the reading difficulty. An indi- 
vidualized plan based on intensive 
tutoring will be drawn up for each 
child. It will be the reading spe- 
cialist's job to insure that the 
child is receiving the reading 
training judged to be appropriate. 

Importantly, the reading tutoring 
will focus on the skills and aca- 
demic areas that the child is exper- 
iencing most trouble with, and 
tutoring material s will be taken 
from the class from which the child 
,^ is drawn for tutoring. Because of 
transportation problems, it was 
judged to be impossible to run an 
after-school tutoring program. The 
reading specialist therefore decided 
to tutor children during classes 
they were already failing (had 
received an "F" in the previous 
report card), reasoning that the 
child could not do any more poorly, 
and it was therefore unl ikely that 
the once-a-week removal for tutoring 
would hurt the chil d's performance 
in that class. The chil d's assign- 
ment for the class he or she is 
missing during the tutoring sessions 
constitutes the material for that 
session. For example, if a cl.ild is 
tutored during his science class, he 
is tutored in reading using the sci- 
ence material his class will be cov- 
ering that day. 

Addition of an Intensive Study Pro- 
j ; ram 

EJLeven students that are experi- 
encing academic failure , who have 
been judged by project and school 
staff to have the requisite ability 
to perform successfully in school , 
and are not enrolled in the MAP, are 
involved in an Intensive Study Pro- 
gram. In this program, they meet 
weekly with a project staffer to go 
over their assignments , pi an their 
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next week^s academic work, and 
receive tutoring. The staffer (who 
is in the building full time) checks 
on their academic progress almost 
daily, and works with the students 
and their teachers to insure that 
the students keep up in their day-- 
to-day assignments. 

The program centers around a 
"time management" approach, in which 
the timely scheduling and completion 
of assignments is emphasized. It is 
hoped that the students will better 
know what is required of them in 
each of their classes, will schedule 
a particular time to do the assign- 
ment, and will complete it on time. 
It is believed that the focused 
attention of a project staffer will 
motivate these students to complete 
their school work and improve their 
grades. 

Additional Project Changes 

In 1980-81 some limited use of 
college student volunteers had been 
made in tutoring, and in supervising 
the in-school suspension center. 
Student volunteers were not used in 

1981- 82 because of problems in 
Supervising the volunteers and in 
insuring that they would be availa- 
ble on a regular basis. For the 

1982- 83 year student volunteers will 
again be recruited, and a more 
systematic effort will be made to 
supervise and coordinate their 
efforts. Volunteers will be primar- 
ily used in helping with extracurri- 
cular activities , in tutoring, and 
in collecting and maintaining atten- 
dance data. 

Due to the OJJDP budget cuts, the 
project was forced to eliminate two 
support staff positions: the atten-. 
dance clerk and the secretary in the 
counseling center. The duties of 
these persons are being covered by 
other project staff , and by the use 
of volunteers. 



The final change in Milwood 
Alternative Education Project activ- 
ities for the 1983 year is one of 
emphasis. Project staff have always 
been concerned with getting Milwood 
students involved in school activi- 
ties and school decision-making. 
They were disappointed in the evalu- 
ation resul ts that indicated that 
the students did not perceive them- 
selves to be any more involved in 
the school or to be influencing 
school policies. Project staff plan 
to Spend more time in the coming 
year working with students in plan- 
ning and carrying out activities . 
They hope to increase the number of 
,^ activities (through planning and the 
use of volunteers) that students can 
become involved in. 

Particular interventions that are 
planned to involve students in 
school activities are a pep club, a 
choir, an intramural program, and a 
spring theatrical^production. ^ These 
programs will be designed and imple- 
mented with the aid of the student 
council and the student advisory 
board, giving these students more 
say in the events at school. 

Recommendations 

1- MAP teachers must be aware of 
the possibil ity of peer influence 
contributing to a negative classroom 
atmosphere. Particularly, disrup- 
tive behavior that is reinforced by 
the other students in the room can 
destroy the possibility of learning. 
This is a drawback to any program 
that Seeks to put a large proportion 
of behavior problem students 
together in one class. Proj ect 
staff have recognized the problem, 
and have atttnipted to rectify it by 
scheduling more "positive" students. 
It is probable though, that the num- 
ber of positive students (two or 
three) in a class of twenty, is 
insufficient to overcome the nega- 
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tive peer influence of the rest of 
the class. The MAP teachers will 
have to be aware of the potential 
problem, and should be prepared to 
respond quickly to contagious beha- 
vioral disruption. 

2. The affective education cur- 
riculum for MAP class, organized 
around a three-day-a-week group 
experience, is not focused, and is 
often carried out in an ad-hoc fash- 
ion. It is apparent that the 
counselors and the teachers develop 
much of the group material on the 
spot. It is unlikely that the 
objectives of the program can be met 
unless specific interventions are 
designed to address .them. 

3. Plans underway to document 
the actual curriculum taught in the 
MAP, should be examined. The 
description of the MAP will only be 
useful to future program implemen- 
ters to the extent that it is clear, 
and covers all activities and inter- 
ventions undertaken. 

4. The teacher of the new remed- 
ial reading program should attend 
closely to possible negative <5ffects 
of taking students out of their 
regular classes. The program does 
plan to organize the reading tutor- 
ing around the material the students 
are covering in their classwork, but 
it is probable that not as much 
material will be covered in the 
tutoring sessions as in the regular 
classes. In addition, the negative 
labeling that could result from 
being sent to this class could harm 
the overall academic performance of 
the students. The reading special- 
ist should make every effort to pro- 
vide tutoring outside of the stu- 
dents' regular class schedule , and 
allow the students to remain in 
their mainstrea.*!! classes as much as 
possible. 

5. The Cambridge-Sommerville 



study (Powers and Wittmer, 1951) 
inVfcSCigated the del inquency-preven-- 
tion potential of assigning a case- 
worker to monitor the behavior and 
provide support to lower class boys 
identif ied at risk for del inquency . 
Long-term follow-ups to that study 
(McCord, 1978) indicate that the 
program actually had severe negative 
effects on the life experience of 
the participants. McCord attributes 
the nega tive effects to a violated 
expectancy that "things are going to 
get better": children' in the pro- 
gram felt "special" and were led to 
believe that their lives would now 
take a dramatic turn for the better. 
, These children found that life did 
not, in fact, get much better. 
Apparently, many of the participants 
internal ized their failures, decid- 
ing that, "Even with all of this 
help, I still can't make it." 
Accordingly, the Milwood volunteers 
must beware of setting up unrealis- 
tic expectations in their clients, 
that, if not met, produce negative 
feelings of self-worth in the chil- 
dren. 

6. The introduction of the in- 
school suspension room did not 
decrease the number of self-reported 
suspensions (which includes both in- 
and out— of-school suspensions). A 
difficulty with in-school suspension 
is that it is a less severe sanction 
than out-of -school suspension, and 
thus, tends to be resorted to for 
less extreme disc'ipl inary infrac- 
tions. Ironically, this "progres- 
sive" pr ocedure can resul t in more 
suspensions. Fortunately, in the 
case of the Milwood proj ect , the 
institution of in-school suspension 
did not increase the number of 
self-reported suspensions and did in 
fact decrease the number of exit 
suspensions — the program's primary 
objective. Nevertheless , project 
staff should make clear to building 
personnel the possibil ity of too 
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frequent suspension, resulting from 
the decreased severity of the pun- 
ishment. Every effort should be 
made to insure that suspensions, 
either in-- or out-of -school , are 
levied only for clearly specified 
extraordinary offenses. 

7. A final caution concerns the 
the Student Council. While it is 



theoretical ly unl ikely that the 
negative effects associated with the 
student council in 1982 were actu- 
ally program outcomes, the possibil- 
ity exists. Project staff must pay 
careful attention to any processes 
that may occur in the student coun- 
cil meetings or in the school at 
large that could produce the nega- 
tive effects found in the 1982 data. 
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Table 1 



Comparisons of Milwood and South on 
Selected 1981 SAES Measures 



Milwood South 



M SD N M SD N 



School variables 



Attachment to school 


.63 


.27 


206 


.65 


.26 


196 


.67 


School rewards 


.20 


.27 


217 


.23 


.11' 


205 


.37 


School punishment 


.20 


.29 


217 


.25 


.26 


206 


.06* 


Parental emphasis on education 


.68 


.26 


218 


.68 


.26 


217 


.92 


School effort 


.57 


.31 


223 


.61 


.29 


219 


.23 


Negative peer influence 


.21 


.2P 


219 


.22 


.23 


217 


.55 


Victimization 


.14 


.17 


211 


.13 


.17 


202 


.52 


Involvement 


.19 


.16 


214 


.22 


.16 


210 


.08* 



Psycho-social variables 



Attachment to parents 


.59 


.28 


222 


.60 


.27 


218 


.87 


Belief in rules 


.68 


.25 


205 


.72 


.24 


187 


.14 


Positive self-concept 


.69 


.17 


194 


.73 


.16 


189 


.03** 


Rebellious autonomy 


.73 


.32 


193 


.63 


.30 


173 


.003** 


Interpersonal competency 


.80 


.23 


204 


.79 


.21 


181 


.69 


Internal control^ 
















Alienation 


.36 


.30 


202 


.37 


.29 


193 


.83 


Practical knowledge 


1.37 


.44 


206 


1.39 


.45 


191 


.66 


Educational expectation 


3.48 


1.72 


222 


3.61 


1.73 


218 


.44 



Self -reported behavior variables 



School non-attendance index 


.54 


.74 


224 


.37 


.68 


218 


.01** 


Delinquency 


.19 


.20 


216 


.14 


.18 


210 


.005** 


Serious delinquency 


.12 


.19 


216 


.10 


.19 


211 


.18 


Suspensions 


.25 


.43 


210 


.14 


.35 


203 


.007** 


Drug use 


.30 


.32 


215 


.17 


.24 


210 


.0000** 


Self-reported grades 


2.67 


.91 


220 


2.69 


.84 


219 


.80 



Background variables 



Sex (% male) 


49% 




221 


51% 




220 


.74 


Age 


13.10 


.87 


225 


13.15 


.95 


221 


.48 


Race (% minority) 


31% 




220 


45% 




209 


.005** 


Parental education 


2.28 


1.15 


196 


2.49 


1.17 


189 


.08 


Reading ability 


1.82 


.87 


200 


1.85 


.86 


201 


.72 



Not available in 1981 . 

* Nearly significant. 

'* Statistically significant. 
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Table 3 



Significant Differences between Milwood and South 
on Selected 1981 SAES Measures 
Controlling for Race and Pariental Education 





Mil wn n H 












Adjusted 
mean 


N 


Adjusted 
mean 


N 


P 


Suspensions 


.25 


181 


.12 


165 " 


.002 


Rebellious autonomy 


.71 


167 


.62 


145 


.01 


School non-attendance 


.52 


'.192 


.38 


176 


.04 


Delinquency 


.19 


184 


.15 


169 


.03 


Drug Use 


.30 


183 


..18 


169 


.000 
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Table 4 



Comparisons of Milwood and South on 
Selected 1982 SAES Measures 



Milwood South 



M SD N M SD n1 



School variables 



Attachment to school 


.64 


.28 


474 


.57 


. .26 


224 


.002** 


School rewards 


.17 


.23 


496 


.19 


.24 


221 


.30 


School punishment 


.22 


.25 


497 


.29 


.30 


221 


.001** 


Parental emphasis on education 


.62 


.28 


407 


.61 


.30 


210 


.68 


School effort 


.58 


.32 


563 


.59 


.29 


233 


.68 


Negative peer influence 


.24 


.23 


547 


.25 


.24 


230 


.58 


Victimization 


.16 


".20 


491 


.21 


.26 


220 


.005** 


Involvement 


.18 


.16 


550 


.21 


.17 


231 


.02** 



Psycho-social variables 



Attachment to parents 


.55 


.30 


558 


.55 


.29 


232 


.99 


Belief in rules 


.70 


.27 


332 


.68 


.26 


191 


.40 


Positive self-concept 


.74 


.18 


384 


.73 


.18 


199 


.53 


Rebellious autonomy 


.63 


.34 


302 


.65 


.32 


183 


.52 


Interpersonal competency 


.78 


.24 


328 


.75 


.25 


188 


.18 


Internal control 


.40 


.26 


346 


.43 


.26 


191 


.22 


Alienation 


.38 


.29 


342 


.43 


.25 


193 


.04** 


Practical knowledge 


1.35 


.46 


295 


1.25 


.49 


186 


.02** 


Educational expectation 


.42 


1.79 


566 


3.42 


1.72 


236 


.72 



Self-reported behavior variables 



School non-attendance index 


.48 


.73 


566 


.56 


.62 


234 


.14 


Del inquency 


.17 


.20 


507 


.20 


.21 


215 


.07* 


Serious delinquency 


.11 


.19 


509 


.14 


.20 


214 


.06* 


Suspensions 


.21 


.41 


459 


.18 


.38 


218 


.36 


Drug use 


.28 


.34 


514 


.27 


.31 


220 


.72 


Self-reported grades 


2.65 


1.08 


565 


2.68 


.95 


230 


.72 



Background variables 



Sex (% male) 


48% 




565 


54% 




235 


.001** 


Age 


13.23 


.93 


573 


13.22 


.90 


235 


.89 


Race (% minority) 


34% 




676 


46% 




308 


.001** 


Parental education 


2,56 


1.21 


511 


2.55 


1.13 


218 


.92 


Reading ability 


1.75 


.88 


516 


2.04 


.84 


216 


.001** 



These are the actual N's. Means and SB's for the South sample are weighted by the 
probability of sampling; all 1980-81 respondents were resurveyed. 
* Strong trend. 
**Statistically significant. 
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Table 5 



Comparisons of Milwood 


and 


South 


on Changes in 


SAES 






Scores from Spring 


1981 to 


Spring 


1982 










Milwood 






South 






M 




SD 


N 


M 


SD 


N 


P 


Changes in school variables 
















Attachment to school -.09 




.33 


69 


-.05 


.37 


68 


.48 


School rewards -.07 




.36 


75 


-.06 


.37 


68 


.85 


School punishment .04 




.42 


.75 


-.04 


.41 


69 


.26 


Parental emphasis on education -.06 




.38 


69 


-.09 


.37 


71 


.60 


School effort -.05 




.42 


81 


-.01 


.40 


74 


.63 


Negative peer influence .07 




.r31 


75 


.01 


.32 


74 


.31 


Victimization .05 




.29 


71 


-.06 


.25 


67 


.82 


Involvement -.02 




.21 


76 


.00 


.25 


70 


.56 



.06 


.43 


83 


-.05 


.42 


74 


.85 


.02 


.36 


59 


-.10 


.34 


55 


.21 


.00 


.27 


58 


.00 " 


"" ".24 


" 60" " 


.98" 


.03 


.42 


47 


.07 


.43 


52 


.25 


.03 


.36 


55 


.00 


.30 


57 


.68 


.11 


.39 


60 


-.01 


.36 


59 


.07* 


.03 


.55 


55 


-.14 


.61 


54 


.32 


.12 


2.45 


84 


-.33 


2.63 


72 


.60 



Change in psycho-social 
variables 

Attachment to parents 
Belief in rules 
Positive self-concept 
Rebel 1 ious autonomy 
Interpersonal competency 
Internal control^ 
Alienation 
Practical knowledge 
Educational expectation 

Changes in self-reported 
behavior variables 
School non-attendance index 
Del inquency 
Serious del inquency 
Suspensions 
Drug use 

Self-reported grades 

Changes in school popula- 
tion ( background ) variables 

Parental education .04 1.71 69 .12 1.62 61 .77 

Reading ability -.13 1.23 67 .34 1.34 65 .04** 



^Not available for 1980-81. 

*Nearly significant . 
**S tatistically significant. 



.07 


1.16 


84 


.19 


1.14 


73 


.51 


.05 


.31 


70 


.05 


.30 


67 


.95 


.03 


.27 


70 


.02 


.30 


67 


.86 


.01 


.63 


70 


.12 


.48 


67 


.27 


.10 


.49 


73 


.12 


.41 


68 


.84 


.15 


1.53 


80 


-.04 


1.36 


70 


.65 
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Table 6 

South Students Performed Better than 
Milwood Students on 1981 M.A.T.s 





Ml 1 LjnnH 

X X JL ^ Vl \J \J Kl 








P 


Adjusted 
mean^ 


N 


Adjusted 
mean^ 




Reading 


32.5^ 


586 


36.3 


571 


.000* 




(44)c 




(52) 






Language 


33.6 


586 


36.6 


571 


.000* 




(44) 




(51) 






Math 


28.4 


586 


31.4 


571 


.000* 




(47) 




(56) 






Composite 


94.5 


586 


104.3 


571 


.000* 




(45) 




(53) 







Note. Results are the same whether using raw scores, percentiles, 
scaled scores, or normal curve equivalent scores. 

a • • 

Means are adjusted for grade, sex, race and age, using SPSS routine 

ANOVA with classical analysis, controlling for school, grade, sex, 
race, and age. 

^Adjusted raw score, 
c • 

Adjusted percentile score. 
*Highly statistically significant. 
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Table 7 



Milwood Students Perform as Well as South Students on 
1982 Reading and Language M,A,T,s, and Peform Better 
than S ou th S tudent s on 19 82 Math M* A* T • s 





Milwood 




South 








Adjusted 
mean^ 


N 


Adjusted 
mean^ 


N 


P 


Reading 


38. 0^ 

(56)c 


5?6 


38.3 

(56) 


580 


.61 


Language 


37.7 

(53) 


57 6 


37.5 

(53) 


580 


.58 


Math 


32.2 

(58) 


57 6 


31.1 

(54) 


580 


.01* 


Composite 


108.0 

(56) 


57 6 


106.8 
(54) 


5 80 


.42 



Note . Results are the same whether using raw scores, percentiles, 
scaled scores , or normal curve equivalent scores, 

a • 

Means are adjusted for grade, sex, race and age, using SPSS routine 
ANOVA with classical analysis, controlling for grade, sex, race, 
and age. 

^Adjusted raw score. . . 

^Adjusted percentile score. 
*Statistically significant. 
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Table 8 



M.A.T. Composite Raw and Percentile Scores for Students of 
Different Ethnicity, Adjusted for Age: Kalamazoo 1981 and 1982^ 



White Black Other 



Adj . Mean° N Adj . Mean N Adj . Mean N 



1981 



1982 



105. 56= 
(55)d 

t 

114.31 

(62) 



770 



765 



84.84 

(34) 

82.69 
(40) 



338 



358 



102.09 
(51) 

96.05 
(55) 



49 



33 



.000* 



.000* 



M.A.T. Composite Eaw and Percentile Scores for Students of 
Different Gender, Adjusted for Age: Kalamazoo 1981 and 1982^ 



Mai e Femal e 

Adj. Mean^ N Adj. Mean N 



1981 97.09 559 101.48 598 .008* 

(47) (51) 

1982 105.81 566 108.89 590 .03 

(53) (56) 



Note. Results are the same whether using raw scores, percentiles, scaled 
scores, or normal curve equivalent scores. 

Race effects are adjusted for grade, sex, and age. Sex effects are adjusted 



for race, grade, and age. 

^Adjusted 
analysis . 



^Adjusted means and 2. levels are from SPSS routine ANOVA, using classical 



c 

Adjusted raw score. 
^Adjusted percentile score. 
*S tatistically signficant. 
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Table 9 



South Students Perform Better than 







Milwoood 


Seventh Graders 


on the 1981 M. 


A.T.s 












Mil wood 






South 










Seventh 


Eighth 




Seventh 


Eighth 








Ad j . Mean 


N 


Adj . Mean 


N 


Adj. Mean 


N 


Adj,. Mean 


N 


P 


Reading 




312 


39.8'' 


274 


31.1 


284 


40.0 


287 


.000* 






(59) 




(43) 




(58) 






Language 


28.8 


312 


40.3 


274 


31.5 


284 


40.5 


287 


.000* 


(35) 




(59) 




(40) 




(59) 






Math 


23.6 


312 


35.2 


274 


26.2 


284 


35.4 


287 


.000* 




(35) 




(65) 




(42) 




(66) 






Composite 


79.8 


312 


115.3 


274 


88.8 


284 


115.9 


287 


.000* 


(34) 




(61) 




(40) 




(62) 







Note . Results are the same whether using raw scores, percentiles, scaled scores, or normal 
curve equivalent scores. Means are computed separately for each school, and adjusted for sex, 
race, and age. Adjusted means are from SPSS routine ANOVA, using classical analysis. £ lev- 
els are for the interaction of school and grade, controlling for school, grade, sex, age, and 
all other two-way interactions. 

^Adjusted raw score. 

^Adjusted percentile score. 

*Highly statistically signficant. 
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Table 10 



Milwood Students Perform as Well as 
South Students on the 1982 M.A.T.s 



Kilvood South 
Seventh Eighth . Seventh Eighth ' 

Adj. Mean N Adj. Mean N Adj. Mean N Adj. Mean N 



Reading 32.7« 292 A5.1 284 31.8 291 A3. 3 289 .39 

(^Tr (69) (AA) (6A) 

Language 33.6 292 A3. 5 28A 32.2 291 A1.3 289 .31 

(A6) (65) (A3) (61) 

Math 28.7 292 37.2 28A 26. A 291 3A.6 289 .13 

(50) (70) (A2) (60) 

Composite 9A.9 292 125.8 28A 90.3 291 119.2 289 .21 

(56) (69) (.A2) 62) 



Note . Results are the same whether using raw scores, percentiles, scaled scores, or normal 
curve equivalent scores. Means are computed separately for each school, and adjusted for 
sex, race, and age. Adjusted means are from SPSS routine ANOVA, using classical analysis. 
£ levels are for the interaction of school and grade, controlling for school, grade, sex, 
age, and all other two-way interactions. 

^Adjusted raw score. 
^Adjusted percentile score. 



*Highly statistically signficant. 
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Table .11 



Average Change in Adjusted M.A.T. Percentile Scores 
from 1981 to 1982 for the Milwood and South 
Seventh and Eighth Grades 







Milwood 






South 






Seventh 


Eighth 


Overall 


Seventh 


Eighth 


Ov er al 1 


Reading 


+12 


+10 


+12" 


+1 


+6 


+2 


Language 


+11 


+6 


+9 


+3 


+2 


+2 


Math 


+15 


+5 


+11 


0 


+6 


+2 


Composite 


+22 


•I-8 


+12 


+2 


0 


+1 



Note . These percentile change scores are adjusted for sex race, and age. The 
interaction of school x year, controlling for sex, race, and age, is signifi- 
cant, p<.001, for all four M.A.T. scores. 
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Table 12 



Comparisons of 
Milwood Alternative Program Students 
to Milwood Eighth Graders 



M.A.P. Eighth Graders 

K SD N M SD N 



School variables 

Attachment to school .62 

School rewards .20 

School punishment .26 . 
Parental emphasis on education .55 

School> effort .48 

Negative peer influence .34 

Victimization .19 

Involvement .17 

Psycho-social variables 

Attachment to parents .48 

Belief in rules .65 

Positive self-concept .68 

Rebellious autonomy .72 

Interpersonal competency .82 

Internal control .51 

Alienation .46 

Practical knowledge 1.34 

Educational expectation 2.76 

Self-reported behavior variables 

School non-attendance index .63 

Delinquency .27 

Serious delinquency .19 

Suspensions .44 

Drug use .41 

Self-reported grades 2.28 

Background variables 

Sex (% male) 49% 

Age 14.00 

Race (% minority) 2S% 

Parental education 1.97 

Reading ability 1.72 



.25 


45 


.61 


.28 


202 


.73 




47 




1 Q 


71 7 
Zi J 


.Uz*^ 


.27 


•47 


.24 


.26 


213 


.53 


26 


49 


SQ 
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. Z 0 
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1 oZ 


.4z 


32 


SI 


SA 
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7^ n 
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.Z J 
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49 
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7^ 1 
Z J 1 




23 




1 s 


70 
. ZU 


71 7 
Zl J 


1 Q 


16 




• 1/ 




77 Q 

zzy 


mil 


.24 


51 


.51 


.30 


231 


.42 




^ s 


• Do 




1 /i 7 


.54 


1 s 


J ? 


79 


.10 


loo 


.19 


. J i. 


J u 


Aft 


. JZ 


1 07 




9A 
. Z'f 


J / 


. / J 


.z4 


1 A A 

140 


.09* 


. Z O 


J O 




o c 
.z!) 


ICO 


.02* 


.26 


38 


.38 


.28 


150 


.10* 


.37 


36 


1.40 


.44 


130 


.43 


.89 


51 


3.47 


1.75 


235 


.01** 


.75 


51 


.52 


.74 


234 


.36 


.25 


45 


.21 


.21 


219 


.10* 


.25 


42 


.12 


.20 


219 


.04** 


.50 


43 


.20 


.40 


204 


.001** 


.40 


43 


.37 


.40 


219 


.04** 


.82 


47 


2.59 


1.10 


234 


.06* 




49 


44% 




233 


.65 


.81 


51 


13.70 


.70 


235 


.009** 




51 


20% 




290 


.08* 


.12 


44 


2.61 


1.12 


217 


.001** 


.86 


46 


1.76 


.86 


223 


.77 



* Almost significant. 
**Statistically significant. 
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Table 13 



Significant Differences between M.A.P. and Milwood Eighth Graders, 
Controlling for Parental Education, Ethnicity, and Age 



M.A.P. Eighth Graders 

Adjusted Adjusted 
Outcome measure mean N mean N p 



School rewards .23 38 .12 186 

Suspensions .41 35 -21 177 



.002 
.01 
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Table 14 



Significant Changes on SAES Scores from 1981 to 1982 for 

M.A.P. Students and Milwood Eighth Graders 

M.A.P. ' Other Eighth Graders 

Outcome measure Mean Change SD N Mean Change SD N p 

ri 

Educational expectation -1.67 2.69 15 .22 2.29 68 .006** 

Negative peer influence .25 .36 11 .03 .29 63 .03** 
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Table 15 



Comparisons of Skills Lab Students 
to Milwood Population 
and to M.A.P. 



Schoo l variable s 

Ati hcient to school 

School rewards 

School punishment 

Parental emphasis on education 

School effort 

Negative peer influence 

Vict imization 

Involvement 

Psycho-social variables 
Attachment to parents 
Belief in rules 
Positive self-concept 
Rebellious autonomy 
Interpersonal competency 
Internal control 
Al iena tion 
Practical knowledge 
Educational expectation 

Self-reported behavior variables 
School non-attendance index 
Del inquency 
Serious del inquency 
Suspensions 
Drug use 

Self-reported grades 

Background variables 
Sex iZ mal e) 
Age 

Race (Z minority) 
Parental education 
Reading ability 



Skill 


s Lab 




Milwood 




P 


M.A.P. a 

P 


M 


SD 


N 


M 


SD 


N 


.43 


.23 


10 


.64 


.28 


464 


.02** 


.006* 


.11 


.18 


9 


.17 


.23 


487 


.44 


.15 


.45 


.32 


9 


.21 


.25 


488 


.005** 


.03** 


.50 


.29 


7 


.62 


.28 


400 


.25 


.77 


.28 


.32 


16 


.59 


.32 


547 


.001** 


.004** 


.56 


.28 


14 


.23 


.22 


533 . 


.001** 


.001** 


.22 


.34 


9 


.15 


.19 


482 


.30 


.77 


.12 


.14 


15 


.18 


.16 


535 


.11 


.17 


.43 


.32 


16 


.56 


• .30 


- 542- 


^.09* 


^ .59- - 


.57 


.19 


5 


.71 


.19 


327 


.24 


.35 


.55 


.21 


5 


.74 


.18 


379 


.03** 


.04** 


.73 


.37 


5 


.63 


.34 


297 


.49 


.93 


.83 


.32 


6 


.78 


.23 


322 


.61 


.82 


.52 


.30 


6 


.40 


.26 


340 


.28 


.94 


.51 


.32 


7 


.38 


.29 


335 


.21 


.58 


1.19 


.28 


6 


1.36 


.47 


289 


.38 


.28 


1.35 


1.70 


17 


3»43 


1.76 


549 


.001** 


.001** 



.94 


.97 


17 


.47 


.72 


549 


.009** 


.02** 


.47 


.28 


12 


.17 


.20 


495 


.001** 


.001** 


.39 


.32 


12 


.10 


.18 


497 


.001** 


.002** 


.88 


.35 


8 


.20 


.40 


451 


.001** 


.01** 


.56 


.37 


14 


.27 


.34 


500 


.002** 


.06** 


1.50 


.94 


14 


2.68 


1.07 


551 


.001** 


.001** 


69Z 




16 


47Z 




549 


.15 


.23 


14.29 




17 


13.20 




556 


.001** 


.09* 


13Z 




24 


23Z 




652 


.46 " 


.13 


1.33 


.86 


15 


2.60 


1,20 


496 


.001** 


.008** 


I.IB 


.81 


17 


1.77 


.88 


499 


.006** 


.008** 



Comparison of Skills Lab students to M.A.P. students. 



* Strong trend. 
**Statistically significant 
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ff^U 16 



fi|Plti«Mi €aiirp«ri»oat of SV.il !• Ub and M.A.P. Students* 
€««tr«lli«s for r»rr«tal Cdtteation, leadiiis Ability and Age 





•Kit la t««D 




MAP. 

Iff* » • 







m 


Acinar 


— 


J %l V <W f7U 








Man 




mean 


R 


P 


AlincNMr^t 10 Kkeol 


.40 


*e 


.68 


30 


.03 




1.14 


IS 


3.13 


31 


.001 




.08 


13 


.21 


31 


.03 




,5b 


12 


.28 


30 


.03 




.36 


14 


.64 


31 


.03 




.45 


1 


.24 


31 


.09 




.47 


10 


.21 


27 


.02 




.62 


12 


.29 


27 


.06 




.41 


10 


.15 


27 


.04 




i.S7 


12 


2.49 


29 


.004 




.96 


6 


.34 


30 


.02 
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Table 17 



Comparison of Quasi-Randomly Selected Student Council 
Students to Students Not Selected for the Student Council 



Student Council Control 
Personal characteristic M SD N M SD 



School variables 

Attachment to school .59 

School rewards .30 

School punishment .21 
Parental emphasis on education .^3 

School effort .50 

Negative peer influence .32 

Victimization .19 

Involvement .27 

Psycho-social variables 

Attachment to parents .43 

Belief in rules .69 

Positive self-concept .66 

Rebellious autonomy .63 

Interpersonal competency .76 

Internal control .49 

Alienation .39 

Practical knowledge 1.34 

Educational expectation 2.60 

Self-reported behavior variables 

School non-attendance index .53 

Delinquency .21 

Serious delinquency .10 

Suspensions .35 

Drug use .32 

Self-reported grades 2.69 

Background variables 

Sex (% male) 55% 

Age 13.27 

Race (% minority) 27% 

Parental education 2,31 

Reading ability 1.70 



.27 


25 


.64 


.29 


151 


.40 


• J 1- 




20 




1 S8 


05** 


.20 


28 


.25 


.26 


159 


.51 


. J ^ 


99 


. Q Q 


9 S 
. Z J 


1 "^7 
1 J / 


OA** 


. J ^ 


9ft 
z o 


S ft 
• JO 


"^7 


1 7 S 
1 / J 


71 
. zi 


7A 


9 

Z J 


7*^ 

. Z J 


9*^ 
. Z J 


17 0 
1 / u 


OA** 


9 9 


9ft 
ZO 


1 ft 


9 9 
. zz 




7 ^ 


. 1 ^ 


9ft 
ZO 


90 
. z u 


.ID 


171 
1/1 


0*^** 


.33 


30 


.56 


.29 


175 


.02** 


7ft 
• z,o 


1 ft 

1 o 


7n 


97 


116 

1 1 u 


Ql 


9'^ 


90 
zu 


7 


1 ft 




OS** 


. J / 


1 7 
1 / 


. 


. J J 


1 OQ 

1 U7 


Q7 


97 
. z/ 


1 ft 
1 o 


7 Q 


9*^ 
. Z J 


1 1 
1 1 J 


1 

. D 1 


9*^ 
. Z J 


1 ft 
1 o 




97 
. Z / 


1 91 
1 zi 




.32 


17 


.38 


.29 


119 


.89 


.47 


17 


1.40 


.44 


106 


.58 


1.87 


30 


3.58 


1.76 


176 


.006** 


.73 


30 


.47 


.73 


178 


.64 


.17 


24 


.19 


.21 


158 


.70 


.12 


24 


.12 


.20 


158 


.73 


.48 


26 


.21 


.41 


147 


.13 


.38 


24 


.31 


.36 


159 


.97 


.97 


29 


2.69 


1.02 


175 


.98 




29 


45% 




173 


.42 


.87 


30 


13.31 


.87 


178 


.78 




34 


22% 




226 


.53 


1.51 


27 


2.66 


1.23 


158 


.20 


.91 


27 


1.82 


.87 


161 


.52 



* Almost significant, 
♦♦statistically significant. 
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Table 18 

Significant Differences between Student Council Members and Other Students 
When M.A.P. and Skills Lab Students are Excluded from the Analysis 





Student Council 


Other 


Students 






M 


SD 


N 


M 


SD 


N 


P 


Attachment to parents 


.44 


.35 


25 


.58 


.30 


151 


.04 


JLllV \J X. V CUlCil c 


.26 




23 


.21 


.17 


147 




Negative peer influence 


.35 


.24 


25 


.22 


.22 


146 


.006 


Parental emphasis on education 


.54 


.33 


17 


.66 


.25 


117 


.09 


School effort 


.48 


.32 


23 


.60 


.32 


151 


.10 


School rewards 


.29 


.32 


23 


.19 


.25 


136 


.08 


Self-esteem 


.65 


.27 


15 


.75 


.16 


114 


.05 


^Included because of its relevance 


to the 


project. 


Not 


statistically s 


ignif icant 


• 
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Project PREP: Second Interim Report 
D. K. Ogawa and G. D. Gottfredson 



Rationale 

Project PREP is continuing its 
effort to reduce high truancy, drop- 
out, and suspension rates, along 
with disruptive student behaviors in 
12 schools in District 9 in Bronx, 
New York. Ultimately, Project PREP 
anticipates that teachers and stu- 
dents will perceive school as a safe 
and viable place for learning. Alt- 
hough the statement of the project's 
theoretical rationale remains essen- 
tially the same as last year's, the 
following ideas have been added: 

1. Lack of opportunities and 
positive experiences create 
fear and frustration in stu- 
dents, lessens stakes in con- 
formity, lessens self-esteem 
and ultimately leads to anger 
and hositility. Creation of 
opportunities, e.g., jobs, and 
positive use of leisure time 
will decrease delinquency and 
decrease the chances of asso- 
ciating with those who do . 
engage in delinquency. 

2. Students lack caring, posi- 
tive, and strong role models 
and a consistent set of firm 
and fair guidelines to follow. 

3. Students lack commitment to 
the conventional norms and 
beliefs of society. 



Although the theories elaborated 
thus far deal with why the problems 
exist in District 9, further elabo- 
ration of prep's action theories, 
i.e., the project's theories which 
will guide their interventions, is 
needed for the third. year. 

Objectives 

The project objectives, gleaned 
from several sources, are summarized 
in Table 1. Drawing from the 
updated theoretical framework, the 
following four objectives were added 
to last year's: (a) reducing nega- 
tive peer influence among program 
participants when compared to compa- 
rable controls; (b) increasing 
opportunities for a successful tran- 
sition to high school and/or a 
career ; (c ) increasing the use of 
fair and consistent school rules for 
program participants; (d) increasing 
students' beliefs in conventional 
rules. This year's "objectives" 
include processes or critical bench- 
marks necessary for implementation. 
Were the project's theoretical 
framework sharpened, a clearer dis- 
tinction could be made between 
ob j ectives and cr itical benchmarks 
for program implementation. 

Intended Interventions 

In addition to the three major 
components in the first year (i.e. , 
the Alternative School, the Citizen- 
ship Clusters, and the Youth compo- 
nent) a fourth intervention, student 
council , was added for the second 
year of operation. The student 
council is composed of eight elected 
representatives from the Alternative 
School and one representative from 
each of the eight cluster schools. 



This second interim report sup- 
plements an earl ier report (Ogawa, 
1982) which should be consulted for 
a basic overview of this project and 
an account of earlier activities. 
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The Youth Director is primarily 
responsible for the student council 
at the Alternative School and the 
two teacher-coordinators are respon- 
sible for the student representa- 
tives at the Cluster schools. The 
council convenes at least monthly to 
learn about decision-making proce- 
dures and developing an agenda for 
initiating school policy changes. 
After a meeting at the Alternative 
School , the Cluster representative 
meets with the rest of the Cluster 
students at his or her home school. 
This intervention is directed at the 
following obj ectives : (a) reducing 
nega tive peer irif luence ; (b ) in- 
creasing the number of active, pro- 
ductive youths, (c) increasing 
self-concept, and d) in-creas-ing 
attachment to school. 

Project Environment 

A major change has taken place in 
School District 9, the implementing 
organization. During the first year 
District 9's Deputy Superintendent 
(DS) acted as project manager. Fol- 
lowing the DS's resignation, the 
administrative responsibility for 
Project PREP was shifted to the Dis- 
trict's funded programs office, and 
the Community School Board has 
become more involved with monitoring 
the project. 

Staffing 

The District 9 Director of Evalu- 
ation and Research and his assistant 
are no longer involved in performing 
evaluation activities. Tlie Commu- 
nity School Board has assigned a 
data collector to assist the project 
director with major evaluation 
activities such as the spring survey 
administration. Although the data 
collector supervised the paraprofes- 
sionals who actually collected the 
data, he did not adopt the role of 
an on-site evaluation coordinator. 
Many of the on-going evaluation 
tasks were performed by the project 



director, a teacher-coordinator and 
the paraprof essionals. 

In addition to the data collec- 
tor, a new Youth Director was hired 
for the second and third year to 
coordinate the Youth Program and the 
student council. 

Program Implementation 

Project PREP has been serving 
students for two years; however, for 
the first half year, th6 project was 
run by a director with a different 
philosophy than the present direc- 
tor. Although the present director 
„has been on staff since April 1981, 
it has only been recently, during 
this past year, that the proj ect is 
moving in the direction that he has 
conceptualized. An example of this 
is the development of curriculum to 
prepare students for options other 
than college. Pianos and other 
musical instruments were- acquired, 
as well as equipment for a photogra- 
phy class and an auto mechanics 
cI :.3S. 

The extent and nature of this 
year's implementation was ascer- 
tained primarily from project 
records , the program development 
worksheets, supplemented with inter- 
views with the project's staff . No 
formal comprehensive monitoring dev- 
ices were developed to record other 
important dimensions of implementa- 
tion such as the degree and extent 
to which proj ect staff have adopted 
al ternative education techniques 
( for example , see Hal 1 & Loucks , 
1977; Leithwood & Montgomery, 1980). 

Alternative School 

Most of the project development 
occurred with this intervention. 
The participants are still seventh 
and eighth graders referred from 
four local junior high schools for 
behavioral problems. However, the 
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emphasis of the intervention now 
includes providing practical experi- 
ences as well as academics* Both 
the career awareness element and the 
transition to high school components 
were specified in the original pro- 
posal, but were not fully conceptu- 
alized or implemented. The career 
awareness element originally 
involved exposing students to guest 
speakers of different professions. 
As implemented in year two, this 
component allows students to get 
experience at the alternative school 
in various activities such as pho- 
tography, auto mechanics, carpentry, 
and electronics. In addition, by 
the end of the year, students were 
exposed to job resumes, participated 
in mock job interviews, and, by the 
summer , 25 students were reportedly 
placed in jobs around the community 
for minimum pay. In addition to 
providing students with vocational 
experiences, the program provides 
students with practical experiences 
such as exposure to museums, restau- 
. rants, ball-games, and taking a 
train trip to other nearby cities. 

English, math, social studies, 
and science are taught in the morn- 
ings. Teachers work with eight stu- 
dents per class for a one-hour per- 
iod* Individualized educational 
plans , based on test scores, were 
initiated for students in December, 
1981, and were updated the following 
April for the remainder of the year. 
Students experiencing academic dif- 
ficulties are also provided with 
tutoring after school. For those 
experiencing emotional difficulties, 
psychological and drug counseling 
are also available from the staff 
psychologist and the local mental 
health cl inic. 

Although an al ternative setting , 
the school is structured as a junior 
high or intermediate school where 



students are prepared academically 
for high school. The project direc- 
tor was concerned, hov/ever, that 
Project PREP students would become 
so acclimated to the supportive 
alternative school environment that 
they would be unable to make the 
transition to high school once they 
graduate from the eighth grade . As 
a consequence , arrangements were 
made with local high school princi- 
pals to allow eighth grade PREP stu- 
dents to attend classes in the morn- 
ing. Five eighth graders were 
placed in high school last February. 
They took courses during the morning 
then came back to the Alternative 
School in the afternoons for tutor- 
ing and counsel ing, in addition to 
participating in the practical cur- 
riculum. 

Both the academic and practical 
components of the Alternative School 
are designed to increase academic 
achievement, self-esteem, attachment 
to school , student involvement, and 
to decrease negative peer influence 
among the targeted students as com- 
pared to comparable control stu- 
dents. In addition to the academic 
and practical components , a third 
component , informal counsel ing , 
designed to increase students'' 
beliefs in conventional beliefs, is 
primarily performed by the project 
director during field trips or other 
activities whenever and wherever a 
topic comes up. Students also call 
the youth director and teachers 
after hours for help. In order to 
provide consistent availabil ity of 
staff to students , arrangements are 
being made to provide a 24-hour 
"hot-line" to students. The project 
director feel s that the exposure to 
different experiences and a suppor- 
tive car ing s t af f ar e the mo s t 
important aspects of the alternative 
school. 
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Citizenship Clusters 

While the project director is in 
charge of the Alternative School, 
the two teacher-coordinators are 
primarily responsible for the Citiz- 
enship Clusters, i.e. , the teachers 
of project students, the paraprofes- 
sional s who operate the Time-Out 
Room, and the students themselves. 
The teacher-coordinators arranged 
weekly meetings with teachers and 
paraprof essional s , coordinated stu- 
dent council activities , provided 
the prizes for the token economy and 
were in charge of evaluation issues 
such as maintaining the design and 
supervising data collection activi- 
ties. 

Unlike last year, the token econ- 
omy went smoothly due to the ready 
availability of the prizes. How- 
ever, the function of the Time-Out 
Room changed from the behavioral 
model of a nonstimulating , "cool 
down" environment for a misbehaving 
student to a microcosm of the "per- 
fect classroom/' Once behavioral 
contracts were established with pro- 
ject students, tne Time-Out Room was 
utilized less. Some principals felt 
that the paraprof essional s were not 
using their time ^ efficiently and 
consequently sugf sted that the par- 
aprof essional s offer tutoring to 
project students. The Time-Out Room 
eventually became £ supportive envi- 
ronment where ^uudents received 
individualized ' itorial and counsel- 
ing servic p 7m the paraprof es- 
sional s, as well as a place to "cool 
down." It appears from subjective 
accounts that the Time-Out Room met 
the needs of students who needed 
personal attention, both academi- 
cally and emotionally, but who could 
not get this from a regular class- 
room- 

One paraprof essional from a jun- 
ior high school describes her role 
as first talking to a student who is 
sent to the Time-Out Room for 



misbehaving. She ascertains why the 
student misbehaved and allows the 
student to vent his or her emotions. 
She then explains why that behavior 
was inappropriate and draws up a 
behavioral contract agreed upon by 
the student , teacher , and parapr o- 
f essional. If the student fulfills 
the contract , he or she receives 
points which are later redeemed for 
prizes. When there are no students 
in the Time-Out Room, the parapro- 
fessional goes to a Cluster class to 
get four students for " reading and 
math tutoring. Each student is to 
be seen by the paraprof essional 
twice each week for rap sessions or 
,^counsel ing. Whether visiting the 
Time-Out Room for tutoring or a 
behavioral probl em, students are in 
the Time-Out Room no longer than 45 
minutes before returning to class. 

Although all paraprof essional s in 
the eight cluster schools do essen- 
tially the same tasks," the intensity 
varies across school s , as can be 
seen in Tabl e 4. For behavioral 
problems, students at one school had 
as few as five contacts whereas at 
another school students had an aver- 
age of 87 contacts . The average 
number of rap sessions each student 
had ranged from two at one school to 
as many as 160 at another school. 
Part of this variation may depend on 
the relationship between the para- 
professional and teacher. Some 
teachers initially f el t threatened 
by the presence of a paraprof ession- 
al providing direct services to stu- 
dents. This was especially true if 
students became attached to the 
paraprof essional s. Other teachers 
excessively use the Time-Out Room to 
unburden themselves from certain 
students. Meetings with the teach- 
ers, paraprof essional s , and the 
staff psychologist were held in an 
attempt to resolve these problems ; 
but, attendance at these meetings 
was minimal • ' 
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In summary, the "Time-Out-Room" 
is a misnotier for the model actually 
being implemented in the Cluster 
Schools. Although it appears that 
the "Time-Cut -Room" has moved away 
from the behavioral model, the ori- 
ginal conceptualization of this 
intervention, as outlined in the 
project's proposal, includes indivi- 
dual ized tutoring and counsel ing 
services. Unclear role expectations 
between teachers and paraprof ession- 
al s and haphazard referral of stu- 
dents at some schools threaten the 
strength and integrity of this 
intervention. Overall, the strength 
of this intervention, according to 
the project director, depends on the 
relationship between the student and 
the paraprof essional . 

Youth Component 

This intervention was originally 
designed to bring Alternative School 
and Cluster students together to 
engage in positive leisure time 
activities. In year two, however, 
the Alternative students and the 
Cluster students became more inde- 
pendent of each other. Alternative 
students plan their own field trips 
separate from Cluster students and 
utilize separate night centers. 
There are 16 night centers in the 
District sponsored by Continuing 
Education which are open from 7-9 
p.m. Only one of these centers is 
run by the Alternative School Staff. 
Alternative students attend this 
center while Cluster students attend 
other nearby centers staffed by some 
of the para-professionals who take 
attendance. However, token points 
are no longer given to students who 
attend the night center. Although 
students are not required to attend 
the night centers, field trips, or 
other activities , they are strongly 
encouraged. Of all the activities 
of the Youth component , the night 
centers are utilized most often. 
Other components include winter 
camp, summer camp and summer jobs. 



PREP 

Student Council 

This was the first year this 
intervention was implemented. Alt- 
hough council members ^were intro- 
duced to parliamentary procedures, 
took field trips to governing agen- 
cies , and addressed school and com- 
munity problems , the proj ect direc- 
tor felt the council was not as 
strong as anticipated. This was due 
in part to the fact that the council 
only met once a month and council 
decisions were often temporary. 

Parent Involvement 

This has been the most difficult 
»j ... 
component to implement. Ministers, 

technical assistance, and consul- 
tants have been ut il ized this year 
in order to organize parents. The 
various strategies were successful 
in involving some parents. 

Implementation Overview 

Project prep's interventions have 
been implemented in varying degrees. 
Tables 3 through 4 provide an over- 
view of proj ect implementation. 
Table 3 shows the average number of 
times alternative school students 
were reported to be involved in var- 
ious activities, and Table 4 pre- 
sents similar information for the 
cluster school s. Implementation is 
vastly different in different 
schools. 

Interim Outcomes 

Methods 

Design . With the first year's 
non--equivalent control group design, 
little could be concluded about the 
effectiveness of Project PREP's 
interventions. Consequently , an 
experimental design was establ ished 
in both the Alternative School and 
the Cluster schools. A pool of eli- 
gible referrals from the four feeder 
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junior high schools were' randomly 
assigned to either the Alternative 
school or a control condition in 
which the students would remain in 
his or her original school. A simi- 
lar procedure v;as conducted for the 
Cluster Schools. Cluster treatment 
students were designated by the PREP 
staff to utilize the Time-Out-Rootn. 
Cluster controls were to be treated 
no differently from non-program par- 
ticipants. 

Due to the mobility of students 
in the Bronx, it was necessary to 
maintain a continuous pool of refer- 
rals. Because some treatment stu- 
dents on the waiting list subse- 
quently withdrew from school, it 
became necessary to develop a treat- 
ment waiting list and a control 
waiting list. Each time a person 
from the treatment waiting list was 
assigned to treatment, a person from 
the control waiting list was also 
added to the control group. Post- 
randomization checks using the SAES 
pretest showed -no sizable differ- 
ences between treatment and control 
students. Larger pools of eligibles 
were developed in some schools than 
in others, resulting in some non- 
proportionality in the design. 

The experimental design was ini- 
tially met with some resistance by 
principals who wanted to place cer- 
tain students in the Alternative 
School and other principals who 
wanted all students served by the 
Time-Out Rooms. Over time, princi- 
pals accepted the randomization pro- 
cess; however, there were a few 
occasions when the project director 
accepted nonrandomized referrals 
into the Alternative School . These 
students are excluded from our ana- 
lyses. Although control students 
repor tedly do not receive any PREP 
services , it is possible that they 
receive other services provided by 
the District. For example, control 
students can attend any of the night 
centers or they may be involved in 



the Gates' program which offers 
remedial reading. In addition, one 
Cluster school has a drug program 
for students and Special Education 
has resource rooms set up at some 
schools to deal with academic prob- 
lems. While Project PREP is unique 
in that it offers both academic and 
behavioral services, there are other 
sources from which control s can 
receive help. 

Subiects . Of the treatment and 
control students, 70% were black and 
30% hispanic. Females made up 33Z 
of the sample while 67% of the sam- 
ple were males. Fifteen students 
returned to the Alternative School 
from last year, leaving 25 openings 
to be filled by students randomly 
selected from a referral pool. Dur- 
ing the year, the project director 
accepted into the Alternative School 
three non-random referral s from 
principal s and one from the proba- 
tion department. Each Cluster 
School served 15 students, with the 
exception of School 148, which 
expanded its program later in the 
school year to 30 students. 

Measures . We intended to rely 
primarily upon the SAES question- 
naire for measures of Project PREP's 
goals and objectives* Official 
school records and the Project's 
parent questionnaire were also to be 
used* The project administered the 
parent questionnaire twice during 
the year to parents of PREP partici- 
pants. 

Data quality . The utility of the 
data available is severely limited 
due to extensive missing informa- 
tion* Required information was not 
provided in sufficiently complete 
form to make most analyses meaning- 
ful . For completeness, tables show 
the actual number of cases for whom 
data are available. In Table 5 we 
illustrate the missing data problem. 
For the Alternative School , there 
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were 21 control and 31 treatment 
students. For the Cluster schools, 
there were 96 control and 162 treat- 
ment students. Complete data on 
withdrawal from school were availa- 
ble. Data on absences from school 
are also reasonably complete. Other 
data are so often missing that r^o 
confidence may be had in analyses of 
the data that are present./ (Table 5 
is illustrative only. Other questi- 
onnaire-based measures resemble the 
measures shown in Table 5 in their 
completeness. ) 

Results 

Proiect PREP^s goals . Table 6 
shows treatment and control group 
comparisons on several measures of 
Project prep's goals. Four measures 
were intended to assess truancy and 
dropout. As noted earlier, however, 
the only sufficiently complete data 
for meaningful analyses are for 
absences (based on school records) 
and withdrawal from school. Stu- 
dents at the Alternative School 
withdrew from school more often than 
controls, but this result could have 
occurred by chance. Treatment stu- 
dents in the Cluster schools with- 
drew from school significantly (p < 
.02) more often than did control 
students. This difference is 

unlikely to have occurred by chance. 

Alternative School treatment stu- 
dents were absent slightly and non- 
signif icantly less often than con- 
trol students. Among cluster school 
students for whom data were 
reported, attendance was better (.10 
< p < .11) for treatment than con- 
trol students. Attendance data were 
repor ted , however, only for students 
who did not withdraw from school . 
Recall that significantly more 
treatment (13%) than control stu- 
dents (4%) withdrew from school. 
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The results shown for class cut- 
ting and school cutting in Table 6, 
although showing significant differ- 
ences, are not very meaningful due 
to the low response rates for these 
questionnaire-based measures. 

. Proiect prep's Objectives . The 
remainder of the results shown in 
Table 6 relate to Project PREP 
objectives. Due to extensive miss- 
ing data, these results are undepen- 
dable. Nevertheless, the resul ts 
suggest that victimiza'tion may be 
si ightly higher among Alternative 
school students than among control s 
(p < .05) , and that negative peer 
,^ influence may be higher in both 
Alternative School and cluster 
treatment groups than in the control 
groups (p < .10) . 

A number of Project PREP objec- 
tives are specific to the Alterna- 
tive School . Resul ts relating to 
these ob j ectives are summarized in 
Tabl e 7 (for obj ectives relating to 



cluster school s was performed sepa- 
rately for each school. Because of 
differences in the size of the pool 
of el igibl es, different proportions 
of students were selected for treat- 
ment in the eight schools. This 
implies that school is ' associated 
with probabil ity of treatment, and 
that differences observed between 
the pooled treatment and control 
groups could be due to school dif- 
ferences as well as to treatment 
differences. Data are not analyzed 
separately by school due to the very 
low statistical power that would 
result from the small sample sizes 
involved. When significant treat- 
ment-control differences were found, 
statistical controls for school were 
introduced to determine whether 
school differences could account for 
the resul t . Such additional ana- 
lyses were performed for withdrawal 
from school and days absent. (These 
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students). As was true for most 
measures summarized in Table 6, 
missing data makes interpretation of 
tlfese results difficult. No signi- 
ficant differences were found bet- 
ween treatment and control groups 
however. 

One way to assess the objectives 
for the Alternative School itself 
(those relating to fair and consis- 
tent rules and to teacher compe- 
tency) is to compare the school cli- 
mate measures bearing on these 
objectives to those for other 
schools and to the Alternative 
School^s standing on these measures 
in the previous year. Chapter 5 and 
the Appendix of Part I of this 
volume show results of the climate 
surveys conducted in the spring of 
1981 and 1982. An examination of 
these results suggests the follow- 
ing: (a) Students report that the 
rules are fairer than do students in 
most other schools, and reports of 
fairness increased somewhat in 1982 
over 1981. (b) In both years stu- 
dents reported that the school rules 
were clearer than did students in 
other schools. (c) In 1981 students 
reported fairly typical levels of 
victimization and safety; in 1982 
students reported significantly more*^ 
safety than in 1981. (d) In both 
years students reported high levels 
of individualized instruction, 
(e) Students reported experiencing 
slightly less disrespect from teach- 
ers than did students in the typical 
school in 1981 , and they reported 



analyses were not" performed for 
questionnaire measures where missing 
data were severe limitations.) 
Large and significant differences in 
withdrawal across schools are pre- 
sent in the da ta. Even when con- 
trols for school are applied, the 
observed difference in withdrawal is 
present and statistically signifi- 
cant (p< .025). 



less still in 1982. (f) More stu- 
dent-teacher interaction than in the 
typical school was reported both 
years , but students reported less 
interaction in 1982 than in 1981. 
(g) In 1981 students reported 
receiving about the same levels of 
punishment in school as did students 
in the typical school , but si ightly 
more than average in 1982. (h) In 
both years students reported receiv- 
ing many more rewards in school than 
did students in the typical school. 

The. results of the parent questi- 
onnaire are shown in Table 8. Alt- 
hough more parents who responded to 
the questionnaire reported visiting 
the school and being aware of vari- 
ous opportunities in the community 
in the second half of the school 
year, the sample may be biased due 
to the low response rate (32 to 
44%). In addition, factors which 
are not controlled for other than 
Project PREP^s interventions , may 
have produced the observed differ- 
ences. 

Discussion 

Limitations 

Problems of missing data ravaged 
the utility of the evaluation. 
Satisfactorily complete data were 
available only for two outcome mea- 
sures: School absences and withdra- 
wal from school . Failure to admin- 
ister the questionnaire to treatment 
and control .students, and poor com- 
pletion rates among students who 
were surveyed inade the questionnaire 
measures only of very limited use- 
fulness. The project-developed-and- 
administered parent questionnaire 
suffered from similar low response 
rates. Finally, information on dis- 
ciplinary referrals was collected in 
such a way that made it impossible 
to distinguish between students with 
no referrals and students for whom 
no data were collected, and were 
therefore unusable. Accordingly, we 
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regard only the absence and withdra- 
wal results as dependable, and focus 
our discussion on those results. 

Withdrawal from School 

A larger proportion of students 
participating in the Alternative 
School than of control group stu- 
dents withdrew from school (19% vs. 
5%, respectively). This result, 
al though not statistically signifi- 
cant, is worrisome. There is no 
evidence that participation in the 
Alternative School reduces the risk 
of dropout . 

A larger proportion of treatment 
students in the cluster schools than 
of control students withdrew from 
school (13% vs. 4%) and this differ- 
ence is statistically signilicont 
when statistical controls for school 
of attendance are applied (p < 
.025). Evidence suggests that par - 
ticipation in the Citizenship Clus - 
ters increases the risk of dropout . 

Absences 

Students attending the Alterna- 
tive School are absent about as 
often as control students. There is 
no evidence that participation in 
the Alternative School reduces 
truancy . 

Treatment students in the cluster 
schools who have not withdrawn by 
the end of the school year are 
absent less than are control stu- 
dents, but this difference is not 
statistically significant. There is 
no evidence that participation in 
the Citizenship Cluster reduces 
truancy . 

Evaluation Strategies for 1982-83 

The Program Development Evalua- 
tion model has never been adopted by 
Proj ect PREP. Clearer specification 
of the proj ect ' s action theory and 
more careful attention to planning 



and management v;ould be useful. 

The PDE managment plan will need 
to be specified more carefully next 
year in order to carry out data col- 
lection activities. PREP showed 
minimal improvement over the first 
year with respect to data collec- 
tion; however, and serious problems 
persist . Identification of obsta- 
cles in this area can be fed into 
renewed planning to strengthen the 
evaluation. 

Th^ process of identifying obsta- 
cles and formulating strategies to 
overcome the obstacles worked well 
^^with implementing randomization this 
year. Randomization will again be 
implemented next year in the Alter- 
native and Cluster schools. Some of 
the Cluster schools will be expand- 
ing the number of students they 
serve from 15 to 30, which will 
require maintaining a large pool of 
referrals. Due to the high mobility 
rate of students, referral pool s 
must be replenished and maintained 
throughout the school year. This 
effort will be of limited use unless 
the completeness of data collection 
improves. 

Problems Project PREP is Encounter - 

Implementation . The project 

director is skeptical about the use- 
fulness of the Time-Out-Room as an 
intervention. He feels the themes 
and prizes may be useful for elemen- 
tary school students but that trips 
and various practical experiences 
may be more valuable with the older 
students . As a consequence, the 
Alternative School and the Clusters 
are separate entities, with the pro- 
ject director responsible for the 
Alternative School and the teacher 
coordinators responsible for the 
Clusters. The project director 
devotes most of his attention to the 
Alternative School. 
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Perhaps a more difficult task for 
PREP is getting parents involved in 
their children's activities. It may 
be, however, that the project is 
dealing with an overwhelming obsta- 
cle in their forcefield — the commu- 
nity environment. Parents in the 
community are often more concerned 
about larger issues such as unem- 
ployment , poor publ ic services , and 
high crime, rather than their chil- 
dren's activities. The community 
environment may also make it diffi- 
cult for the proj ect to reduce del- 
inquent behavior in the schools and 
among PREP students if the surround- 
ing environment supports this behav- 
ior • 

Evaluation issues . The project 
did not devote sufficient personnel 
resources and attention to collect 
the data completely, despite the 
hiring of an on-site data coordina- 
tor. Part of the problem also is 
the quality of records kept at some 
of the schools. The way question- 
naire administration has been plan- 
ned for and executed have been seri- 
ous defects which must be remedied 



if the effects of the project are to 
be determined. 

Successes Proiect PREP is Experienc - 
ing 

Although the objective evidence 
does not depict Project PREP as suc- 
cessful , subj ective evaluations of 
the project are more promising. The 
proj ect is praised in the District ; 
other schools in the district want 
to be served by PREP. After a visit 
to the Alternative school, one staff 
psychologist was so impressed with 
the students, she sought air time on 
a radio broadcast so students could 
„ tell about their experiences . 

Teachers and principals in PREP 
schools were initially skeptical , 
but now thay want the program 
expanded. Some students come to 
school early, and do not want to 
leave after school is_^out. _Some 8th 
graders do not want to graduate. 
Non-PREP students at the Cluster 
Schools have asked how they can join 
Project PREP and participate in the 
activities and in student council. 
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Table 1 



Project PREP' 



s Goals and Associated Objectives 



Goal I: Reduce truancy and dropout in thg 12 participating schools. 
Objectives : 

1. Increase parental involvement and enhance the extent to 
which they value education. 

2. Increase the number of active and involved students. 

3. Increase academic achievement and success. 

4. Increase student commitment/attachment to school. 

5. Increase chances for a successful transition to high school 
and career. ^ 

6. Increase student self-concept. 

7. Reduce negative peer influence^ 

8. Increase teacher competencies so they can cope with problem 
behaviors and meet students' needs. ^ 

9. Increase students belief in conventional rules. 

Goal II: Reduce the amount of disruptive behavior in the 12 schools. 
Objectives: _ ^ 

1. See objectives 1-9 for Goal I, above. 

2. Increase Lne use of consistent and fair guidelines for 



Goal III: Reduce the number of suspensions in the 12 schools. 
Objectives : 

1. See objectives for other two goals (objectives are the 



behavior. 



same) . 



a, 



This objective is specific to the Alternative School. 
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Table 2 
Average Number of Times 
Time-out Room Utilized During the 1981-1982 
School Year for Each Cluster School 



School Behavioral Rap Sessions, 

Problems Counseling 
M SD N M SD N 



55 21.9 25.6 17 3.2 3.2 17 

63 44.7 7.7 16 89.3 7.7 15 

64 86.6 17.8 14 160.5 18.6 13 
82 4.9 6.9 22 5.4 14.5 21 

117 41.9 14.7 16 56.9 30.0 16 

132 13.8 12.1 19 2.3 2.4 19 

147 30.1 33.4 18 94.5 68.3 18 

148 10.4 9.7 39 8,0 7.8 39 



For all schools 26.62 28.4 161 41.5 26.4 158 
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Table 3 



Average Number of Times Alternative School Students 
Participated in FREP Activities During 1981-82 School Year 



Activity 




M 


SD 


Evening Recreation 


28 


.0 


21.7 


Tutor ing 


11 


.1 


8 .6 


Student Council 


2 


.1 


4.0 


Field Trips 


17 


.3 


11.9 


Drug Counseling 


8 


.0 


8.1 


Volunteer Work 


10 


.3 


19.3 


Parent Visits 


4 


.1 


4 .6 


Practical Curriculum 


40 


.2 


26.3 


Winter Camp' 


0 




0 



Note , Non-randomiEed returning students are included; 
N = 46. 



Table 4 



Average Number of Times Cluster Students Participated in PREP Activities 

During the 1981-82 School Year 



Evening Recreation Field Trips Student Council Tutoring 



School 


M 


SD 


N 


M 


SD 


N 


M 


SD 


N 


M 


SD 


55 


0 .0 


0.0 


17 


3.1 


1.2 


18 


0.1 


0.2 


18 


0.0 


0.0 


63 


53.9 


12.5 


15 


1.4 


2 .0 


17 


0 .6 


2.4 


18 


24.3 


7 .7 


64 


82.1 


20 .0 


14 


1.0 


0.0 


14 


0.0 


0.0 


14 


102.1 


30.2 


82 


1.4 


3.6 


22 


0.2 


0 .4 


22 


0 .1 


0.6 


22 


0 .3 


1.1 


117 


a 


a 


a 


1.0 


0 .0 


16 


0.1 


0.2 


17 


49.9 


16 .1 


132 


0 .5 


0.7 


2 


2.0 


0.0 


18 


0 .2 


0.7 


19 


0 .8 


0.5 


147 


46.3 


27.5 


18 


0.9 


0.3 


18 


4 .4 


2.5 


18 


57.9 


45.2 


148 


0.6 


0.5 


40 


0.5 


0 .8 


40 


0 .1 


0.3 


40 


0 .7 


0 .2 



^Information not available. 
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Table 5 

Illustrative Percentages of Data Available 



_ Alternative School Cluster Schools 

Measure Treatment Control Treatiuent Control 



GPA 


61 


48 


76 


71 


Withdrawal 


100 


100 


100 


100 


Absences 


90 


90 


84 


78 


Negative peer influence 


54 


38 


55 


53 


Attachment to school 


48 


... 28 


41 


41 


Interpersonal competency 


39 


19 


31 


29 


Self-concept 


39 


19 


25 


26 


Delinquent behavior 


45 


19 


48 


44 
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Table 6 



Treatment-Control Group Comparisons for Project PREP: 
Overall Project Goals and Objectives 



Alternative School 



Cluster Schools 



Treatment 



Control 



Treatment 



Heasure M SD 



SU 



Control 



SU 



Truancy and dropout 
withdrawals^ 
pays absent^ ^ 
Class cutting 
School cutting 

Disruptive behavior 

pelinqucnt behavior 
Victimization 

Suspensions 

project records 
Self-reported 

Other objectives 

parental emph. educ. 
Negative peer infl. 
Involvement 
Self-concept 

Interpersonal competency 
Attachnent to school 
GPA^ b 

Grades , 
Reading ability 



.19 .^0 

9.91 13.15 

.72 1.41 

.44* .92 



.26 

.25** 



.47 



.54 
.43* 
.24 
.67 
.63 
.73 
66.08 
2.44 
1.69 



.18 
.27 



.30 
.22 
.18 
.15 
.30 
.20 
16.08 
1.15 
.79 



31 
28 
18 
18 



14 
15 



.52 15 



13 
17 
17 
12 
12 
15 
19 
18 
16 



.05 
11.53 
1.11 
1.33 



.26 
.02 



.50 



.42 
.25 
.23 
.69 
.85 
.75 
56.58 
2.00 
1.75 



.22 
14.98 
1.05 
1.66 



.15 

.05' 



.55 



.34 
.25 ■ 
.23 
.22 
.19 
.18 
21.96 
.92 
1.04 



21 


.13** 


.34 


162 


.04 


.20 


96 


19 


18.40 


26.97 


137 


25.48 


34.50 


75 


9 


.81* 


1.29 


107 


.45 


1.04 


60 


9 


.50 


.99 


105 


.38 


.94 


60 


A 


.17 


.21 


78 


.14 


.16 


42 


7 


.22 


.24. 


64 


.21 


.25 


39 




2.40 


5.22 


15 


3.12 


7.36 


8 


6 


.38 


- .49 


60 


.30 


.46 


37 


6 


.63 


.29 


54 


.64 


.24 


32 


8 


.28* 


.21 


89 


.22 


.14 


51 


9 


.25 


.19 


87 


.24 


.21 


51 


4 


.66 


.16 


40 


.67 


.18 


25 


4 


.76 


.20 


50 


.70 


.26 


28 


6 . 


.55 


.28 


67 


.62 


.26 


39 


10 


65.93 


14.11 


124 


72.59 


43.95 


68 


8 


2.39 


1.07 


102 


2.43 


.84 


58 


*8 


1.41 


.78 


94 


1.50 


.91 


54 



Based on school or project records. 



Based on SAES student questionnaire. 

* p < .10 
** p < .05 
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Table 7 



Treatment-Control Group Comparisons for Project PREP: 
Objectives Specific to the Alternative School 



Alternative School Control 

Measure M SD N M SD 



Increase successful transi- 
tion to high school or 
career 



Worked for pay last week .59 

Hold full/part-time job .18 

Educational expectation 3.53 

Practical knowledge 1.21 

Increase belief in 
conventional rul es 

Bel ief scale .59 



.51 


17 


.50 


.53 


8 


.53 


17 


.25 


.46 


17 


1.50 


17 


3.33 


1.87 


9 


'^34 


11 


1.34 


.48 


5 


.26 


11 


.70 


.22 


5 



Note . No significant differences between treatment and control group were 
found on any measures summarized in this table. 
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Table 8 

Responses on Project PREP'S Parent 
Questionnaire Administered in Winter 1981 & 
Spring 1982 for Alternative & Cluster Schools 



Item 


Alternative School 

Winter 1981 Spring 1982 
I responding t responding 
favorably 3 favorably b 


Cluster School 

Winter 1981 Spring 1982 
I responding I responding 
favorably'^ favorably^ 


Attended one or more parent association meetings Ikl 


hit 


khl 


ni 


Aware of reading and math clinic 


53 


100 


59 


82 


Aware of rec. centers 


47 


100 


29 


78 


Aware of night centers 


29 


100 


21 


58 


Aware of comunity agencies 


18 


33"- 


15 


53 


Visited school one or laore times 


65 


67 


71 


88 


Helped child with homework once/week or more 
School meiets child's needs 


47 


67 


76 


62 


76 


100 


78 


82 


Positive change in child's behavior 


71 


100 


62 


88 


Positive change in child's attitude 


65 


■ 100 


53' 


90 


Positive change in child's school atten, 


71 


100 


60 


82 


Wants child to continue in PREP 




100 




93 


Response rate: 


m 


32^ 




39X 



Note : Respondents were parents who had children participating in Project PREP. 



aN=17. 
• %=12. 
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The Jazzmobile Alternative Arts Education Project 



D. E. Rickert, Jr. 



The Jazzmobile Alternative Arts 
Education Project ^s aim was to uti- 
lize the arts as a medium through 
which juveniles can constructively 
channel their energies. The project 
was intended for juveniles in sixth, 
seventh, and eighth grades who 
showed disruptive behavior, who were 
chronic absentees and truants, or 
who experienced academic failure. 

Problems, Goals and Theoretical 
Rationale 

Problems 

The project^s grant proposal 
cited community crime and delin- 
quency, as well as high rates of 
school suspensions, truancy and 
dropping out as problems on which 
the project would focus. In its 
proposed work plans the following 
problems were to be addressed: 

1. Youth hanging out on the 
streets and being involved 
with what project staff called 
"street activity ." Students 
who hang out on the streets 
rather than go to school were 
seen as more likely to get 
into trouble with the law. 

2. Youth not having the opportu- 
nity to live up to their full 
potential to be successful in 
life. 

3. Lack of parental awareness of 
their children's school-re- 
lated activities and lack of 
involvement in these activi- 
ties. 

4. Truancy and chronic absentee- 
ism. 



Goals 

The project's goals were, there- 
fore, to reduce truancy and chronic 
absenteeism, reduce youth involve- 
ment in "street activity," increase 
parental awareness and involvement , 
and increase positive utilization of 
students' potential.. Table 1 sum- 
marizes these goals and indicates 
how the first goal was measured. 
Measures of the other goals were not 
identified. 

' Theory and Obiectives 

The theory of action in the pro- 
j ect' s Program Development Evalua- 
tion (PDE) worksheets is less far- 
reaching than that elaborated in the 
original grant proposal and in early 
meetings between project and evalua- 
tion staff. The theory appearing in 
the Program Development worksheets 
appears to have been a compromise 
resulting from staff discussion sub- 
sequent to the first meeting between 
proj ect staff and evaluation staff. 
Several staff members mentioned that 
such revision took place because 
some of the wording of the original 
theory sounded belligerent. For 
instance, the theory originally con- 
tained a statement to the effect 
that the school systems were 
designed for middle-class white 
youths and that the potential of 
minority children is not being real- 
ized beccuse parents do not use 
their power to change the school 
system. Key staff made it clear on 
several occasions that the "offi- 
cial" action theory (i.e. , the one 
elaborated in the PDE plan) was not 
theirs nor were they free to revise 
or elaborate on it. 

The grant proposal de scr ibe d a 
comprehensive ecological theoretical 
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pespective. It stated that economic 
impoverishment impinges on the fami- 
lies of many youths in Central Har- 
lem. Unemployment, poor housing, 
community crime, health problems 
associated with poverty, and poor 
schools contribute to school fai- 
lure. Dropout and subsequent unem- 
ployment can then introduce a host 
of other problems — namely crime, 
drug abuse, and alcoholism. 

Figure 1 illustrates my interpre- 
tation of the relation between 
(a) the theory described in the 
grant proposal , (b) the theory as 
laid out in the proj ect 's PDE woi: It- 
sheets, and (c) ideas discussed 
informally by project managers and 
staff. Elements enclosed in boxes 
are those included in the PDE work- 
sheet. The central element is un - 
tapped youth potential . As Figure 1 
shows, youth potential may not be 
realized for various reasons. But 
if it is not, students become bored 
with traditional schooling. This 
leads to school failure, which leads 
to frustration. Frustrated students 
are more likely to be chronically 
absent from school and hang out on 
the streets. This leads to delin- 
quency . 

Figure 2 is an abbreviation of 
Figure 1 that illustrates the goals 
and objectives of the project as 
laid out in the PDE worksheet. The 
elements enclosed in boxes were con- 
sidered goals by the project. That 
is , they were regarded as valuable 
in their own right, and are not sim- 
ply instrumental objectives. 

Objectives 

Table ^ 2 shows the ojectives 
(i.e., intermediary outcomes) iden- 
tified by proj ect personnel and the 
corresponding measures. As was the 
case with goals, many of the pro- 
ject's objectives were not measured 
despite some initial plans to pro- 
vide these measures. 



Interventions Intended 

The art classes . The primary 
intervention planned was the Jazzmo- 
bile Art classes. Most students in 
the program were to attend daily 
classes in either drama, music, 
dance or visual arts during the 
regular school day. In the 

intermediate school, six artist-in- 
structors were to teach four classes 
daily. An After School /Sinnmer pro- 
gram was planned to serve 60 
intermediate school students , who, 
for one reason or another, could not 
attend classes dur ing the regular 
school day. The design of the Jazz- 
mobile Alternative Education Arts 
classes was consistent with Jazzmo- 
bile's theory of delinquency preven- 
tion: Attachment between students 
and ^ artist-instructors would be 
fostered by small class sizes (e.g. 
less than 10 students per class) and 
warm, caring artist-instructors who 
coul^d relate l:o the^ student's ,~ espe- ' 
cially in terms of cultural experi- 
ences. The artist- instructors were 
to serve as conventional role models 
to the students. 

The artist-instructors were to 
make an effort to provide new exper- 
iences daily for students and 
encourage creative expression to 
keep students excited and stimu-- 
lated. Black cultural history was 
to be incorporated into classes as 
seemed appropriate to the individual 
artist- instructors. Performances 
and shows were seen by Jazzmobile 
staff as alternative rewards (to 
grades) to help make school a more 
desirable place for students. Other 
alternative rewards planned were 
Jazzmobile tee-shirts, certificates 
of participation, and art-related 
field trips. 

Perhaps most importantly, 
artist- instructors were to increase 
the extent to which students value 
education by teaching discipl ine 
through the arts. Jazzmobile staff 
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maintain that art requires a discip- 
lined process that is transferable 
to academic matters. 

According to the original plan 
artist-instructors were to enhance 
the school staff^s approaches to 
teaching. Conversations with Jazz- 
mobile staff members imply that 
intuition was to play a large part 
in this enhancement effort and its 
assessment • 

Curriculum Guides 

Jazzmobile intended to develop a 
curriculum guide for each of the 
four areas for which there was a 
Jazzmobile class: drama, music, 
dance, and visual arts. These 
guides were intended to assist the 
Jazzmobile artist-instructors in 
providing an artistic experience for 
students within the traditional aca- 
demic learning process. Specifi- 
cally, the guides used the specific 
art form which each guide deals with 
as a vehicle for teaching selected 
content area, skills in language 
arts, social studies, science and 
math. 

It is not clear what objectives 
in the Program Development Plan the 
curriculum guides sought to achieve, 
but presumably these guides would, 
through a "trickle-down" effect, 
raise students^ self-esteem and 
interest in academic subjects. 

Counseling 

A Jazzmobile counselor was 
assigned to the intermediate school . 
This counselor's mission was to 
counsel students regarding careers 
in the arts and to intervene when 
behavior problems arose. The 
counsel or was al so to act as a 
resource consultant and take stu- 
dents on field trips. The objec- 
tives of these activities were to 
increase youth's career awareness. 



primarily in the area of art, 
secondarily in other areas , and to 
reduce youths' sense of hopeless- 
ness. 

Parent Component 

The parent component was intended 
to promote increas£?d attachment bet- 
ween parents and their children 
through their involvement in educa- 
tional activities. Home visits by a 
Field Worker , and parent workshops 
given by the staffs of ' the project, 
the school , and the pol ice precinct 
were to be included in this compo- 
nent . 

Project Environment 

Implementing Organization 

Jazzmobile began in 1965 as a 
non-profit organization giving free 
musical performances, principally of 
jazz. 

Dr. William Taylor, better known 
as "Billy" Taylor, the jazz pianist, 
was a leader in the formation of the 
Harlem Cultural Council, which 
obtained an initial $10,000 grant 
from Ballantine Beer to put on free 
concerts in Harlem. Jazzmobile was 
incorporated in 1966. Over the 
years , Jazzmobile has become 
increasingly involved in music and 
cultural education. The other fine 
arts (drama, dance, and visual arts) 
are now included under Jazzmobile' s 
purv iew. 

The overarching mission of Jazz- 
mobile is the preservation and pro- 
pagation of Jazz. This involves 
presenting free musical performances 
to., the community, particularly areas 
deemed to be culturally deprived. 

Secondary aims are the following: 
(a) to stimulate residents, espe- 
cially youth, into a more active 
participation in musical perf or- 
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mances; (b) to demonstrate to 
students the importance of communi- 
cation and cooperation through the 
arts; (c) to bring students into 
personal contact with artists they 
can relate to and emulate; and 
(d) to provide for students a 
"built-in success factor," whereby 
through- the arts youngsters can 
experience success , develop their 
talents, channel their energies, 
acquire self-discipl ine , develop 
good study habits , and develop 
respect for their school as a vital 
part of their community. 

Internal Organization 

The Jazzmobile Alternative Educa- 
tion Arts Program was one of several 
activities with which Jazzmobile, 
Inc. was involved, but during its 
operation was a substantial source 
of income for the umbrella agency. 

Figure 3 describes the organiza- 
tional structure of Jazzmobile. 
Presiding over all Jazzmobile, Inc. 
affairs is a volunteer Board of 
Directors. An Executive Director 
(ED) has sole responsibility for the 
operation and fiscal management of 
all Jazzmobile activities, including 
the Alternative Education Arts Pro- 
gram. The ED consulted with an 
advisory committee for the Alterna- 
tive Education Arts Program, which 
held monthly meetings. 

Jazzmobile's Development Officer 
(do) is directly under the Executive 
Director (ED) in the organizational 
structure. The DO authored the 
alternative education project's 
grant proposal, and served under the 
the ED as "Education Specialist" on 
the project . The latter involved 
consulting as needed with the ED on 
"educational matters." 

Also officially directly under 
the ED were the Project Director 
(PD) and Evaluation Cowrdinator 
(eg). Informally, they appear to 



have received ongoing supervision by 
the DO. In practice , there appears 
to have been some ambiguity about 
how much authority the Proj ect 
Director and Evaluation Coordinator 
had to make decisions without the 
ED's approval. This arrangement may 
have caused difficulty in executing 
evaluation-related activities 
because of ambiguity about appropri- 
ate channels for organizing the 
work. The official organization 
chart shows no direct link between 
the PD and the EC. 

Directly under the Project Direc- 
tor were the project's Guidance 
,^ Counselor, Curriculum Coordinator, 
Field VJorker, and the Artist-In- 
structors. 

The Interorganizational Environment 

Jazzmobile, Inc. , is well known, 
1 ocal ly and na t ional ly , and ha s 
informal links with other organiza- 
tions, especially arts organizations 
in the Harlem community. These 
linkages include those with the 
local police precinct (i.e.. Pre- 
cinct 28), the Upper West Side 
Health Center, local banks, the Har- 
lem Urban Development Corporation, 
the Goddard Community Center, the 
Dance Theatre of Harlem, Harlem Cul- 
tural Council, Upton Chamber of Com- 
merce , Children's Art Council , and 
Studio Museum of Harlem. Represen- 
tatives of some of these organiza- 
tions serve on either the Jazzmobile 
Board of Directors or served on the 
Alternative Education Arts Program 
Advisory Committee. Jazzmobile's 
President was a Council Member of 
the National Council for the Arts, a 
Presidential appointment . He was 
also President of the Arts and Busi- 
ness Council of New York City. The 
membership of this organization 
includes representatives of 60 
national and multi-national corpora- 
tions (e.g., Exxon, AT&T, Mobil) and 
60 arts organizations. Jazzmobile 
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also has longstanding links with the 
Community School Board in the target 
school district. These links have 
been an asset in Jazzmobile'^s previ- 
ous arts and cultural projects. 

With regard to the Alternative 
Education Arts Program, and the 
evaluation of the program, two cri- 
tical links were weak: (a) that 
with the principal of the project^s 
primary target school , and (b) links 
with parents and other residents in 
the area of the school. 

Staffing and Resources 

The Jazzmobile Alternative Educa-- 
tion Arts project consisted of the 
following staff positions: 

Executive Director/Project ^tan- 
ager 

Project Director 

9 Artist-Instructors — School 
Day Component 

4 Artist-Instructors - After- 
School /Saturday and Summer Com- 
ponents 

1 Evaluation Coordinator 

1 Guidance Counselor 

1 Field Worker 

1 Curriculum Coordinator 

1 Clerk-Typist 

1 Accountant 

1 Secretary 

1 Audiovisual Production Special- 
ist 

Job descriptions of the above 
staff were provided by the project 
and are included in Appendix A. 
Table 3 summarizes project staffing 
by primary work location and educa- 
tional background. As of June 1979, 
15.11 full-time equivalent (FTE) 
workers were employed by the pro- 
ject. Of this, 7.00 FTE workers 
were primarily stationed at the 
Jazzmobile office and 8.11 FTE work- 
ers were primarily stationed in the 
schools. Of the 35 people ever 
emp^loyed by the project, 11 (31%) 
had left the project^s staff by June 
1982. 



The Communitv 

The community targeted for ser- 
vices by the Jazzmobile Alternative 
Education Arts Program is called 
Central Harlem. The population of 
Central Harlem is almost entirely 
black. Reportedly, the unemployment 
rate for black youths in Central 
Harlem is about 60%, while it is 18% 
for adults (over 21). The income 
level in Harlem is low: 1979 sta- 
tist^' cs indicate that 50% of fami- 
lies were earning less- than $5,000 
(grant proposal , p. 13) and a large 
percentage of Harlem residents 
receive some type of public assis- 
tance (e.g., welfare). 

Local health care facilities are 
described as inadequate. Infant 
mortal ity is high. The housing 
stock is reportedly decl ining by 
3000 units per year due to arson and 
abandonment (New York Times, 197 8). 
The specific target area as 
described by Jazzmobile staff is 
even worse than Central Harlem at 
large. Crime and delinquency rates 
are among the highest in New York 
City. Police data indicate that the 
precinct within which the project 
operates ranked 11th out of 73 pre- 
cincts for detention and arrests of 
youth between the ages of 7 and 20. 

The Schools 

Three schools in the school dis^ 
trict TTere targeted by the Alterna- 
tive Educa tion Ar ts Program : An 
intermediate school and two elemen- 
tary schools. The intermediate 
school was the pr imary target , as 
interventions in the elementary 
schools were minimal. The elemen- 
tary schools are two of four feeder 
schools for the intermediate school . 
According to Jazzmobile administra- 
tors the intermediate school was 
chosen by the Community School Board 
because of the serious problems with 
which it is plagued. 
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WItifin 9t the frftWtt 

Ift lfH« Jacwobile was awarded a 
ftra«l Mftdtr Iba bM»rtraqr School Aid 
Act (E»AA) Title ?ll to Ivplenent an 
Arte Cittiiaitimi CarUhmnt Frotraa in 
aMiber aclio>al dtetrict in Cast Bar* 
Im. The ir.teat vat to lend support 
ta tba 4i»trict^a deMtrecation plan 
if atl«»ptint to avtliorate the 
atfteraa effects of sinority grou? 
iaelatioa* Jassnobile received 
anettHir CSAA trant in 1977 to vork 
in Ccmunity School District 3 
iVf^t tfest Side) and 4 (Cast Har- 
lan)^ Nsf^ttan. Hie orta^^^tion 
received still another CSAA grant in 
I97t to continite its activities. 

In tf7S« JatKMbile hecaae inter- 
ested in delinquency prevention and 
diversion for **first ti»e of fen-- 
dera**^ It vas ai^arded a (rant froa 
the Im foforceoent Assistance 
AdsinifCration (UAA) to initiate a 
prevention and diversion progrsn in 
an elfMntary school in the district 
vimre this project now operates. 
Hie project conducted during the 
If7t»7f sclMol yoor called Young 
Cllltena of Rarlesi Art rrogra»» or 
CIIAf. Yhere vera four basic inter- 
vene ion component s t An in»scbool 
arte education enrichaent prograsii 
an af ter*»scliool and Saturday program 
for flrsc*tlM offenders^ vorkahops 
for parontst and a auMser program* 
Stnrfenta f ron grades 3 to 6 partici* 
p#tod» Like the Alternative Cduca-* 
tion Arts prograsi* aajor «pbasis 
MS placed on helping children to 
Vaporlen^e success by channel ing 
tkeir ulenta and energies into the 
aria'' (JaiLiuu AlilltiBfii Ittlt 
April IfSOH Police Precinct 28 
psrtlclpatt4 by assisting in con- 
ducting arts vorkshops and exhibit- 
lAg students^ art work in the police 
station* Students gave perforwances 
mi 4laplay#d ort Mrk at h^psitals 
mi ctiurckes* 

An evaluation of Project OIAP 



reported a reduction in drug-related 
problens and absenteeism, and 
parents reportedly became highly 
involved in the program ( Justice 
Assistance News . April 1980). 

Planning for the Alternative Edu- 
cation Arts Program began in early 
Harch, 1980. A meeting was held on 
March 3, in which Advisory Committee 
members frco the Community School 
Oistr let , key personnel from the 
School District, police, and commu- 
nity representatives met with Jazz- 
siobile staff. On March 29th, an 
abstract of the proposed program by 
Jatraobile was submitted for review, 
discussion, and approval by members 
of the Community School Board, Dis- 
trict administrators, and community 
and business leaders. The proposal 
was approved. The target schools 
were apparently decided upon at the 
first planning meeting. 

Between ^ the time ~ the Community 
School Board approved the program 
and when the program began opera- 
tion! there was an election and the 
Board composition changed. The new 
Board fired the Superintendent and 
installed an acting Superintendent. 
The acting superintendent did not 
achieve permanent status until 
Pebruary, 1982. 

The principals of the target 
schools were apparently told that 
the Alternative Education Arts Pro- 
gram would operate in their schools 
at the beginning of the 1980-81 
school year. The principal of the 
intermediate school was reported to 
have resented the imposition of the 
program* As a result, the principal 
merely tolerated the Jazzmobile pro- 
ject in her school. The evaluation 
of the project fared worse, as the 
principal was opposed to it • The 
basic issue seems to have been a 
complete lack of trust in evaluators 
who aire ^'outsiders J' For a period 
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of time, this distrust extended to 
even the Evaluation Coordinator, a 
Jazzmobile employee. 

The principal's distrust may have 
been fostered by accounts of the 
evaluation of the earlier CHAP pro-- 
ject. An article about CHAP 
appeared in the Justice Assistance 
News (April, 1980). It was entitled 
"Jazzmobile; A Ray of Hope in the 
Crime-Ridden Streets of Central Har- 
lem." The first sentence of the 
article read: "The deck is stacked 
againt the kids who attend New 
York's Public School 113." In the 
body of the article Jazzmobile was 
described as about the only positive 
thing that children in the school 
had going for them. This was seen 
as a personal affront to the princi- 
pal of the school as well as being 
insensitive to the positive aspects 
of Central Harlem. 

Because the Community School 
Board in power when Jazzmobile began 
operation was not the same one that 
helped design the program, there was 
little support from Board members in 
overcoming the obstacles that the 
principal presented. 

What Was Implemented ? 

The First Year 

Basically , we know little about 
what was implemented by Jazzmobile 
during its first year. We do know 
that classes were hel d, and that 
artist-instructors were present in 
the school . Access to all data was 
denied by the school . We were 
advised not to visit the intermedi- 
ate school during the project's 
first year of operation, aid little 
monitoring of the implementation of 
the project's activities was possi- 
ble. 

During the first year of opera- 
tion the Jazzmobile project faced 
numberous obstacles to implementing 



its program in the primary target 
site. Many of these problems seem 
to have arisen from Jazzmobile's own 
poor relationship with the princi- 
pal . These problems were probably 
aggravated by attempts to evaluate 
the program. 

Jazzmobile had problems with 
scheduling and even finding rooms to 
conduct classes. Also the under- 
qualif ication of some of the Jazzmo- 
bile staff fostered resentment of 
them by regular teaching staff. 

The Jazzmobile counselor employed 
during the first year was unable to 
,^ establ ish cooperative arrangements 
with the regular school guidance 
counselor and was denied access to 
students' school records due to a 
lack of proper cx*edential s. During 
the second year she was replaced by 
ano ther counsel or . 

The Jazzmobile field worker had 
problems getting the cooperation of 
parents to participate in project 
activities. This lack of coopera- 
tion may have been related to 
attempts to evaluate the project. 
The PTA president opposed an outside 
evaluation and may have generated 
some distrust of the project by 
parents. 

Despite the foregoing problems , 
the project was able to implement 
its program in some fashion: 
classes were held, some field trips 
were carried out , and some perfor- 
mances took place. 

The Second Year 

By the beginning of the second 
year agreement was reached between 
the evaluators and key project staff 
that they would make a serious 
effort to document plans to overcome 
obstacles in their f orcef iel d and 
document implementation through the 
Program Development worksheets. 
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This' agreement was made during a 
conference in August, 1981. 

In conversations with Jazzmobile 
staff members, it became apparent 
that intuition rather than data was 
to be the technique for measuring 
the degree of implementation. We do 
not know whether artist-instructors 
were "warm and caring," how reward- 
ing the art shows and field trips 
were, or to what extent artist- in- 
structors worked v/ith staff teachers 
to enhance their approaches to 
t each ing . Pro j ec t per sonnel 

resisted the quantification of these 
implementation standards. 

Shortly after the August, 1981, 
conference, the project director 
resigned. The Development Officer 
of Jazzmobile stated that this had 
little impact on the program, as she 
was quickly replaced by a project 
director who was a social worker 
rather than an artist. 

Training of Artist-Instructors 

A problem during the first year 
of program operation was that Jazz- 
mobil e management had 1 i t tl e know- 
ledge of what the artist-instructors 
were actually doing in their classes 
and why. It became apparent at the 
end of the year that some of the 
artist instructors were not doing a 
good job. Also, many of the 
instructors were apparently not 
aware that they were empl oy ed by a 
demonstration project whose primary 
purpose was to prevent delinquency. 

To amel iorate these problems, 
more attention was given to staff 
training during the second year. 
Training began with a formal orien- 
tation week in September, 1981. 
Noel Day of Polaris Research and 
Development (the technical assis- 
tance contractor) participated in 



this orientation. Areas covered 
were the following: (a) Personnel 
matters (sick days, vacation, etc.), 

(b) system change strategies, 

(c) rationale of data collection, 

(d) the national scope .and demons- 
tration nature of the "Delinquency 
Prevention Through Alternative Edu- 
cation" initiative, (e) the Program 
Development Evaluation process and 
(f) long and short range goals of 
the project weekly unit plans. 

For the second year-, a require- 
ment was implemented that artist-in- 
structors submit "unit plans" on a 
regular basis . These plans were 
supposed to describe what artist-in- 
structors planned to do and why . 
They were intended for quality con- 
trol or monitoring purposes. 

After the initial orientation, 
artist-instructors in the day pro- 
gram were required to attend weekly 
staff "meetings 'with" ~the'~ Project 
Director, during which additional 
training took place. Topics covered 
throughput the year included: 
weekly unit plans , guidance con- 
cerns, special events schedul ing and 
planning (e.g. art shows, field 
trips, etc. ) , curriculum, student 
council activities, protocol for 
interaction with parents, protocol 
for interaction and sharing ideas 
with regular staff teachers, and 
student placement concerns (in sec- 
ondary schools) . 

When the after school and Satur- 
day program started in November, 
1981 , the participating artist-in- 
structors met twice weekly with the 
Proj ect Director. Training focused 
on adolescent development , behavior 
modification techniques, special 
needs of gifted and talented stu- 
dents and referral of students for 
special high school placements . 
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Diplomacy; Meeti ti&s with Key Actors 
in the Forcefield 

During the first year, many prob- 
lems arose that strained relations 
between Jazzmobile and the adminis- 
trators of the target schools, par- 
ticularly the intermediate school. 
During the second year, Jazzmobile 
made some effort to anticipate prob- 
lems before they occurred and to 
approach key actors in order to 
overcome obstacles. Many meetings 
occurred during the 1981-82 school 
year, but the following appear to 
have been most significant: 

o Late August, 1981: The Exe- 
cutive Director and Evalua- 
tion Coordinator met with 
Dr. Jerry Evans, President 
of the Community School 
Board - District 3. The 
meeting concerned matters 
affecting implementation of 
the second year program. As 
a result of this meeting, 
the Community School Board 
assigned a staff member to 
act as liaison between Jazz- 
mobile and itself. 

o Sept., 1981: The Jazzmobile 
Evaluation Coordinator met 
with the principal of the 
intermediate school in an 
attempt to make peace, so 
that constructive conversa- 
tion regarding the evalua- 
tion could take place. 
Reports indicate that the 
meeting was cordial, but it 
did not lead to data being 
released for evaluation pur- 
poses. A decision was made 
by Jazzmobile to wait until 
the principal retired in 
November, 1981 before data 
collection would be 
attempted. (The principal 
did not actually retire 
until February, 1982). 

o September l7, 1981: 

-293 



Jazzmobile Evaluation Coor- 
dinator (as acting project 
director ) , the Jazzmobile 
Curriculum Coordinator and 
the newly assigned Community 
School District 3 liaison 
met with the assistant prin- 
cipal at the intermediate 
school to schedule Jazzmo- 
bile classes into the school 
curriculum. 

o Early November, 1981: The 
newly hired Jazzmobile 
Counselor held an initial 
meeting with the school 
staff counselor to introduce 
himself and explain his role 
in th e Jaz zmob il e pr ogr am . 
Reportedly , the new counse- 
lor was able to strike a 
working relationship with 
the school counselor and 
they held regular meetings 
thereafter. Unlike the pre- 
vious Jazzmobile Counselor, 
the new counselor was 
granted access to student 
files. 

o I ^cember 2, 1981: Jazzmo- 
bile Executive Director, 
Evaluation Coordinator and 
new Proj ect Director met 
with the Community School 
Board to discuss data col- 
lection issues. At that 
meeting, the Board decided 
(after 1 1/2 years) that 
Jazzmobile would need affir- 
mative parental permission 
for collection of data from 
school records. 

o December 18, 1981: Another 
Community School Board meet- 
ing was held so that the 
publ ic coul d respond to the 
data collection issues. At 
this time the previously 
passed resolution that Jazz- 
mobile must obtain parental 
permission was upheld. 
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o January 31, 1982; Project 
director met with school 
personnel to coordinate sec- 
ond seme s ter of the Jaz zmo- 
bile Alternative Education 
Arts Program in order to 
adjust to cl ass shifts as a 
resul t of the semester 
change. 

o March 22, 1982: Jazzmobile 
Executive Director, Project 
Director and new Evaluation 
Coordinator met with the 
Community School Board and 
school administrators to 
discuss objections that the*' 
school personnel had to the 
SAES survey • The evaluation 
staff's principal investiga- 
tor and another staff member 
^ also attended that meeting. 
The Community School Board 
decided that the self-report 
delinquency section of the 
student survey must be 
deleted in order for it to 
be administered. 

Public Relations 

The Jazzmobile Alternative Educa- 
tion Arts Program suffered from a 
lack of credibility during its first 
year. School staff reportedly saw 
the arts as an unnecessary "frill," 
The artist instructors did not 
receive the respect from other 
teachers they felt they deserved, 
and this made it harder to do their 
jobs , 

During the second year Jazzmobile 
devoted many resources to showing 
that Jazzmobile did provide a worth- 
while service and that it knew what 
it was doing. In the case of • the 
Project Director and Counselor, this 
involved hiring people with more 
impressive credentials. 

During the second year, a Jazzmo- 
bile Alternative Education Arts Pro- 
gram brochure was issued (late 



November, 1981 )# The brochure gave 
a job description of each staff tnem** 
ber and listed their career accom- 
plishments. The point was to estab-^ 
1 ish the staff members as traitied 
professional educators. The bro-- 
chure was distributed widely. 

Another activity which Jazzmobile 
engaged in heavily during the second 
year was what st^iff called "creative 
documentation" of program impletnen-* 
tation. 

By . mid-October, staff had pre-- 
pared a slide presentation of yeat 
one activities for showing at parent 
meetings and workshops with regul at 
school staff. By the end of Novetn-^ 
ber, 1981, a presentation was ready. 
A video tape of classroom activities 
was ready by J^iiuary , 1982. These 
slide and video Presentations were 
used for training purposes during 
weekly staff meetings and were sh^^n 
at parent ahd^te^cher" wo^rkshopsT 

Jazzmobile al so organized atl 
alternative education resource book-» 
case, published one issue of a nevfS-^ 
letter, and helped the intermediate 
school develop a public relations 
booklet and a yearbook. 

The Art Classeg 

Most sixth graders at the two 
elementary school s participated iti 
arts instruction provided by Jazztno-* 
bile staff. In the intermediate 
school, most (except those retained) 
who participated as 7th graders dur- 
ing the 1980-81 school year were 
continued in the program as 2th 
graders. New 7th graders were 
sel ected in cooperation with 
iutermediate scb^ol staff according 
to scheduling convenience and per-» 
haps other unknown criteria. Sctie-^ 
cules for participating students 
vere completed by mid-September, 
1981, Classes began September 24, 
1981 , at the two elementary schools 
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and September 28, 1981, at the 
intermediate school. While the sem- 
ester was in progress 33 students 
were lost from the program in the 
intermediate school, ov/ing to a Pro - 
motional Gates Program whereby stu- 
dents more than two years behind in 
reading or math were required to 
receive remediation. 

Six artist-instructors worked in 
the intermediate school-day program 
and three artist-instructors in the 
elementary schools. Students in the 
elementary schools attended two to 
three classes weekly. Participating 
students at the intermediate school 
attended Jazzmobile classes daily. 

During October, 1981, four 
artist-instructors were hired to 
teach after-school and Saturday 
classes. Many of the after-school 
students were referred by Police 
precinct 28 or the intermediate 
school as chronic absentees and 
"first time offenders." Classes 
started on November 27, 1981 with 45 
students. Classes met two school 
days a week from 3:15 - 5:00 p.m. 
and on Saturdays from 9:00 - noon. 

By the third quarter of the 
school year 60 students were in the 
afternoon and Saturday program. 
Students in the afternoon and Satur- 
day program were given the opportu- 
nity to continue during the summer 
months (i.e. July 1, 1982-August 27, 
1982). Summer classes met daily 
from 9: 00 a.m. - 3 : 00 p.m. Seven 
Artist /Instructors served 84 stu- 
dents. Classes were held at one of 
the primary schools. Eight interns 
from the New York City Summer Youth 
Employment division of the Depart- 
ment of Empl oyment hel ped out . 
Arrangements were made for students' 
lunches through the Free Lunch Pro- 
gram. 

It is difficult to capture what 
went on in the Jazzmobile Art 
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classes except to say that the 
artist- ins true tors taught arts. It 
is apparent that much more actually 
took place but this was of an ad hoc 
nature and depended greatly on the 
style of the individual artist-in- 
structor. Several artistic accounts 
of instructors' style have been pro- 
vided by the project and are 
included in Appendix B. 

Other Activities 

Besides time in' Jazzmobile 
classes, target students partici- 
pated in a number of out-of -cl ass 
activities such as field trips, art 
,^ shows and performances. 

Field trips . Attempts to conduct 
field trips during the first year 
were ad hoc. Lessons were learned 
concerning the steps necessary in 
the planning of field trips: 
Licensed school staff must be pre- 
sent as cHaperones" (tHis "^requires 
finding cooperative staff), and 
parental consent forms are neces- 
sary . 

A calendar for field trips was 
put together by the Jazzmobile 
Counselor for the second year. The 
counselor also compiled a list of 
students who would be participating. 
With this information in hand, the 
counselor and project director were 
able to make arrangements with 
parents and school staff. 

Al though an expl ana t ion of how 
each field trip fit into individual 
students' treatment plans was not 
available to the evaluators, an 
effort was reportedly made to 
arrange f iel d trips that were a 
meaningful supplement to each stu- 
dent's program* For instance , stu- 
dents who were interested in attend- 
ing city-wide arts high schools made 
field trips to such places. In all, 
31 field trips were conducted, with 
students going to the museum. 
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library, movies, plays, monuments, 
and parks. 

Films were used during the second 
year for the purpose of reinforcing 
concepts covered in Jazzmobile 
classes. Some of the films dealt 
with the arts and some with Afro-Am- 
erican history. Eighteen different 
films about such topics as Paul 
Robison, the Harlem Globetrotters, 
Adam Clayton Powell, Roy Wilkens, 
and sex education) were shown. 

Productions and art shows . Stu- 
dents in the Jazzmobile classes were 
"rewarded" by having the opportunity 
to display their talents. Toward 
this end, 10 productions and arts 
shows were held. Failure to obtain 
proper permits halted plans for a 
street festival at which Jazzmobile 
students were to perform. 

Special events . Several special 
events were carried out as part of 
the Jazzmobile program. These 
included performances by the Nanette 
Beardon Dance Company and a lecture 
by Jazz musician Jimmy Owens. 

Rewards 

All Jazzmobile target students 
received a Jazzmobile tee-shirt and 
a certificate of participation. 
Students involved in final produc- 
tions had cast parties. 

A Jazzmobile Youth Council was 
establ ished at the intermediate 
school. Meetings of elected offi- 
cers began in December, 1981 , and 
took place weekly thereafter. One 
issue of a Newsletter was published 
by the council in January, 1982. 
The newsletter covered information 
about the activities of Jazzmobile 
participants, as well as articles of 
interest to the general student 
body. Seven Jazzmobile participants 
received scholarships to dance 
classes at Harlem School of the 



Arts. Three graduating students 
from the Intermediate School 
received the Chancellor Memorial 
Award, a city-wide recognition for 
musical prof iciency . 

Jazzmobile participated in the 
graduations held at the target 
intermediate school and two feeder 
elementary schools. Jazzmobile Spe- 
cial Art Awards were given to a num- 
ber of graduating students. In 
addition al 1 students received cer- 
tificates of merit. 

Parent Component /Fieldwork 

The Jazzmobile Alternative Educa- 
tion Arts Program emphasized parent 
involvement in their children's pro- 
gram activities. Many parent work- 
shops were held throughout the sec- 
ond year. In these workshops , 
parents were told about the art-re- 
lated activities that their children 
learned during the day . Video-tape 
and ^'^ 'de presentations of perfor- 
manc s wc;;:i ^hown st these meetings. 
The Jazzmobile Field Worker was 
active at soliciting parental parti- 
c ' , a tion. Flyers were sent home 
wi students , phone calls and home 
visits werr made, and appeals were 
mace to the respective Parent Asso- 
ciations . On one occasion a heal th 
fair was l.eld to attract the atten- 
tion of jarents. Also, in October, 
19£ Jazzmobile staff held evening 
hours during "Open-School Week" at 
all three schools so that parents 
who could not attend workshops coulcf 
at least meet the artist-instruc- 
tors. Narratives about the progress 
of the program were mailer' 
parents in December, 1981. 

A total of 74 parent workshops, 
were held between September, 1911, 
and June, 1982. Each workshop had 
between 30 and 35 participants. All 
told, over 105 parents wera consid- 
ered regular participants . Some 
parents were enl is ted to h^jlp carry 
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out seme of the productions put on 
by Jazzmobile. 

Counseling Component 

The Jazzmobile management hired 
an experienced and properly credent- 
ialed counselor in October, 1981. 
The new counselor was better able to 
implement the program in the school 
because he held a degree (i.e. MSW) 
and credentials that were respected 
by regular school personnel. He was 
allowed access to confidential 
records and was effective in soli- 
citing the cooperation of regular 
teachers for such things as chaper- 
oning field trips. 

Besides counseling students for 
"behavioral problems" on an "as 
needed" basis, he engaged some stu- 
dents in career counseling. He 
orchestrated the previously men- 
tioned field trips to arts-high 
schools and prepared an arts career 
directory. The new Jazzmobile 
Counselor established a referral 
procedure for one-to-one counseling 
and organized a case record system. 
He designed special referral forms 
and followed a set appointment sche- 
dule. 

Curriculum Guides 

During the life of the Jazzmobile 
project, two curriculum guides were 
completed and a third started. The 
first (drama) was completed at the 
end of the first year. Little is 
known about how this guide was actu- 
ally used except that there was a 
formal presentation of the guide to 
school administrators in September 
and October, 1981. A second (dance) 
curriculum guide was developed dur- 
ing the second school year in coop- 
eration with the Jazzmobile dance 
instructors and some regular school 
teachers. Training workshops for 
regular school teachers in the use 
of the dance guide were held in late 
May, 1982. 
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Evaluation of the Project 

No evaluation design v/as 
described in the project's grant 
proposal. Subsequent interviews 
with project staff members indicated 
that they expected a simple pre-post 
summative evaluation involving read- 
ily available aggregate data with no 
control group. Jazzmobile had been 
exposed to this type of evaluation 
in its previous youth programs 
(e.g., ESAA and Project CHAP). Sev- 
eral Jazzmobile staff members sug- 
gested that they would not have 
accepted the grant had they known of 
the evaluation requirements. 

The rigorous evaluation — not 
anticipated by Jazzmobile or the 
school syi^tem — created problems for 
Jazzmobile. The building adminis- 
trator of the primary target school 
was vehemently opposed to the pro- 
posed evaluation^ actijj^ities, and 
this may have created difficulties 
in implementing other aspects of ' the 
project . A fair summary of evalua- 
tion-related activities is that they 
were unproductive. The Jazzmobile 
pr o j ect is a good exampl e of a pr o- 
ject that is not amenable to rigo- 
rous evaluation. Jazzmobile staff 
did not see the value of rigorous 
evaluation, Evaluators were told 
time and time again by key staff 
members that the Jazzmobile project 
needed a more "creative" evaluation. 
The fact is, however, that Jazzmo- 
bile was funded as a delinquency 
prevention demonstration proiect and 
needed to be evaluated as such, in a 
rigorous manner. This end was never 
accomplished. What follows is an 
account of efforts to evaluate the 
Jazzmobile project. 

The First Year 

Because the Jazzmobile project 
had already begun operation when 
evaluation efforts began, the evalu- 
ation design was a non-equivalent 
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control group comparison (i.e. 
Jazzmobile target students vs. the 
rest of the students in the 
intermediate school). Evaluations 
of the after-school component and of 
the program in the elementary 
schools were never accomplished 
because problems with the evaluation 
of the major component — the 

intermediate school operation — de- 
voured the entire evaluation effort. 

Although assignment to Jazzmobile 
classes for the 1980-81 school year 
was not random, the assignment was 
fairly haphazard, based on schedul- 
ing convenience. On all variables 
for which data were provided, the 
treatment and non-equivalent control 
group look similar. . Differences 
between groups on age, sex and race 
are non-significant. Nevertheless, 
any attempt to interpret first year 
results must be done with caution 
due to potential bias of an unknown 
nature owing to the following: 
(a) only students for whom parental 
permission was obtained were allowed 
to be surveyed, (b) the difference 
in non-response rate to items on the 
survey between treatment and non-e- 
quivalent controls was highly signi- 
ficant (p<.001). Non-response to 
questionnaire items was far greater 
for Jazzmobile treatment' students 
than for non-equivalent control stu- 
dents. 

The only data available for the 
first year cohort are survey mea- 
sures — no data from records were 
ever released to the evaluators. 

Treatment-control differences on 
all measured outcome variables were 
non-significant, suggesting that the 
program made little difference for 
these outcomes. See Table 4. 



Year Two 

After attempts to create a stron- 
ger design for the second year 
failed, the design again defaulted 
to a . non-equivalent control group 
comparison. The original assignment 
to treatment and control conditions 
was based on scheduling convenience 
for the school. Entire classes were 
designated as either Jazzmobile or 
non- Jazzmobile classes. Then after 
the school year began, the school 
received the students'' heading (CAT) 
scores from the previous school 
year. Those who did not "pass" were 
reassigned to remediation programs , 
and they were not allowed to parti- 
cipate in the Jazzmobile classes. 

Very late in the school year, the 
Community School Board decided to 
rel ease data only for students for 
whom affirmative parental permission 
had been obtained. As expected, 
affirmative permission response rate 
was low, 64% (116) for Jazzmobile 
participants, and 39% (67) for non- 
participants (i.e. non-equivalent 
control s ) . The extent of the bias 
introduced into the evaluation 
design by the shuffling of students 
based on CAT test scores and the 
decision regarding parental permis- 
sion remains unknown. 

The SAES Student Survey, with 
del inquency items deleted, was 
administered in May, 1982 to treat- 
ment and non-equivalent controls for 
whom affirmative parental permission 
had been obtained in the intermedi- 
ate school. 74% (135) of this 
select group responded to the sur- 
vey. The 135 students for whom Stu- 
dent Questionnaire data were availa- 
ble represent 38% of the total 
sample in the intermediate school 
(N=351). No attempt was made to 
survey the 214 seventh graders in 
the two feeder elementary schools. 
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After the end of the school year, 
California Achievement Test scores, 
attendance and promotion data from 
records were provided for the treat- 
ment and control students for whom 
affirmative parental permission had 
been obtained. Data on withdrawals 
were provided for all students in 
the intermediate school • Whatever 
bias was introduced, the treatment 
and control students allowed to par- 
ticipate looked remarkably similar 
on background variables. There were 
no statistically discernable differ- 
ences between treatment and controls 
on participation in special educa- 
tion (t=1.02, p<.31), sex (clii- 
squared=2.66 , dF=l, p<. 10) , self-re- 
ported parental emphasis on 
education (t=-.13, p<.90), self-re- 
ported parental education (t=-.53, 
p<.59), and pre- treatment standard- 
ized achievement test (CAT) reading 
(t=-.68, p<.50) and math (t=.91, 
p<.37) scores. There was no signi- 
ficant difference in racial composi- 
tion between the treatment and non- 
equivalent control groups (t=1.42, 
p<.12). The treatment group did 
differ from the control group on 
grade level. The non-equivalent 
control group was split about 
equally between 7th and 8th graders. 
The treatment group was weighted 
heav ily with 8 th graders ( t=3 .44 , 
p<.01). 

Questionnaire measures * The only 
significant differences found bet- 
ween treatment and non-equivalent 
control groups were on the measures 
of Attachment to School and Student 
Employment. Table 5 shows that the 
treatment group was more attached to 
the school than the control group 
(<.05), and members of the control 
group more often reported holding a 
regular part-time or full-time job. 
If the 24 measures reported in Table 
5 were independent, one would expect 
to find one difference at the .05 
1 evel by chance al one . Among the 
non-significant comparisons, the 
differences between treatment and 
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control groups appear small relative 
to th e St andar d dev i a t i on s . No 
pre- treatment measures usable as 
covariates for attachment to school 
were available, so it was impossible 
to attempt to adjust for potential 
pre-existing group differences for 
this outcome. For student employ- 
ment , grade level was correlated 
both with treatment status and 
employment status. Using grade 
level as a covariate does not alter 
the statistical conclusion. 

Measures from school records . 
The project provided data on with- 
drawal s from school ( for reasons 
,^ other than transfer) for all stu- 
dents in the sample. Other data 
(standardized achievement test 
score s, attendance, and promotion) 
were provided only for students 
whose parents consented, and only 
for students still in school at 
yearns _end._ 

No significant differences were 
found between treatment and control 
groups on math or reading achieve- 
ment, or on school attendance . 
Furthermore, the differences 

observed appear small relative to 
the standard deviations. (See Table 
6.) 

A larger proportion of treatment 
than non-equivalent control group 
students were promoted to the next 
grade (93% vs. 81%), and this dif- 
ference was significant at the p<.05 
level. But the treatment and non-e- 
quival ent control groups al so dif- 
fered (p<.01) on previous promotion 
history and the control group con- 
tained more special education stu- 
dents. When previous retentions and 
special education status were used a 
covariates (to statistically "con- 
trol" for these pre-existing group 
differences^ no significant effect 
of treatment or promotions was found 
(p=.7 8). 
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Jazzmobile participants were less 
likely to withdrav; from school for 
reasons other than transfer than 
v;ere non- equivalent control group 
students (5% vs 13%), No pre-treat- 
ment or other data were available 
for students who withdrew, making an 
attempt to apply statistical con- 
trols impossible. The observed sig- 
nificant difference therefore may be 
a treatment effect, or it may 
reflect pre-existing differences 
between the two groups. We have no 
way to disentangle these competing 
possibilities. 

Discussion 

Limitations of the Evaluation 

This evaluation, which was con- 
ducted under difficult circums- 
tances, has some important limita- 
tions. In the interest of clarity, 
the limitations themselves are 
described here. Later, some of the 
factors contributing to these limi- 
tations will be discussed. 

This has been a quasi-experimen- 
tal evaluation of an intervention 
which, given the range of interven- 
tions possible in schools, was not 
particularly strong on an a priori 
basis. The non-equivalent control 
group design used is imperfect 
because, despite some evidence that 
treatment and control students were 
not grossly different on available 
measures of student background char- 
acteristics, the treatment and con- 
trol groups may have differed in 
unknown ways. Furthermore , data 
were generally available only for 
students whose parents gave affirma- 
tive consent for the release of 
data, and response rates to the sur- 
veys were not perfect , And, most 
information about student outcomes 
was available only for students who 
did not withdraw during the school 
year (the exceptions is student 
withdrawal, which was available for 



all students). These limitations 
made statistical controls for pre- 
existing differences dif f icul t or 
impossible to apply. Biases of an 
unknown nature may influence the 
pattern of results. Finally, no 
direct measures of delinquent behav- 
ior (the primary outcome of interest 
to the project'^s sponsors) was avai- 
lable in the second year of proj ect 
operation. 

Despite these limitations, it is 
possible to say with 'considerable 
assurance that if the project had 
any sizabl e ef f ect s on any of the 
outcomes studied, such affects would 
,^be detectable with the design used. 
This is so primarily because little 
difference was found between treat- 
ment and control students on the 
background characteristics that were 
available for study, and because the 
sample sizes are sufficiently large. 

Limitations of the Project 

A number of influences in the 
proj ect 's environment created limi- 
tations for the implementation and 
evaluation of the proj ect , On sev- 
eral points the proj ect'' s analysis 
of its f orcef ield appears to have 
been incomplete, leading to subse- 
quent problems. For example, that 
Jazzmobile staff working in the 
school might be considered unqual i- 
f ied due to a lack of conventional 
educational credentials was . not 
anticipated. The resulting non-ac- 
ceptance of staff in the school, and 
the influence that the perception of 
non-acceptance had for the day-to- 
day work of the artist-instructors 
and the counselor, was not therefore 
anticipated, 

A second example involves diffi- 
culties in implementing the evalua- 
tion component , The proj ect tended 
to focus on the immediate obstacle 
of the intermediate school princi- 
pal's reluctance to participate in 
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the evaluation, and planned to move 
ahead with data collection after the 
principal had retired (an eventual- 
ity that was considered imminent). 
No plans were made for an alterna- 
tive strategy to obtain data should 
the principal not retire, nor were 
strategies developed to move c.head 
with other aspects of evaluation 
activities when the initial obstacle 
was overcome. 

The Puzzle of Community Support 

Project managers viewed a lack of 
parental concern with and commitment 
to education as a problem, as ' is 
illustrated by their inclusion of 
these parental characteristics in 
their theory statement. Indeed, one 
component of their intervention was 
the series of parent workshops 
described earlier. Yet the commu- 
nity was not successfully mobilized 
to support the evaluation of the 
project. This lack of community 
support for evaluation is illus- 
trated by an important meeting 
called by the chairman of the school 
board to discuss evaluation plans 
with the Jazzmobile and national 
evaluation staff. This meeting, 
which was to seek community input 
for a school board decision about 
the spring 1982 student survey, was 
attended by rel atively few parents 
of ch il dr en in the int ermedia t e 
school where the survey was to be 
performed, and was dominated by 
vocal individuals opposed to the 
survey. An analysis of the force- 
field would have suggested the 
advisabil ity of ensuring, that sup- 
porters of the Jazzmobile project 
attend this meeting in large num- 
bers. That meeting led to the 
school board's censorir^; self-re- 
ported delinquency measurer from the 
survey administration. We are puzz- 
led by the failure of the implement- 
ing organization to mobilize suffi- 
cient community support for the 
project and its evaluation to over- 
come this obstacle, especially when 



the project appears to have had a 
reasonably strong program of parent 
workshops and a full time community 
field worker. 

Principal Support and the Level of 
Delinquent Behavior 

As noted earlier, one potential 
explanation of the intermediate 
school principal 's resistance to the 
evaluation activities was the por- 
trait of East Harlem contained in an 
evaluation report for" an earlier 
Jazzmobile project. That report (as 
did the Jazzmobile grant proposal 
for the present proj ect ) character- 
ized youth in the community as high 
in delinquency and from disorganized 
family and community situations. We 
are somewhat puzzled by the paradox- 
ical lack of agreement between some 
common stereotypes of East Harlem 
and the Jazzmobile grant proposal's 
description on the one hand^ and the 
1 evel of sel f-r epor ted del inquency 
reflected in the 1981 student survey 
results on the other. The level of 
self-reported delinquent behavior in 
our survey was not' especially high. 
Although survey non-response in 1981 
was high, implying that substantial 
biases may exist in the results, the 
results do not support an impression 
of an extremely del inquent popula- 
tion in the intermediate school. We 
see at least two potential explana- 
tions for this paradox. First, 
either response bias or inval id 
reporting may have produced mislead- 
ingly low estimates of delinquent 
behavior among members of this popu- 
lation. Second, the degree of indi- 
vidual delinquent behavior among the 
youths of this community may be 
exaggerated in popular stereotypes 
of the community. Whichever of 
these possibil ities is nearer the 
truth , cl early much remains to be 
learned about the youth of this com- 
munity. The difficulties of con- 
ducting evaluation research in the 
community pose a challenge for 
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responsible service providers and 
researchers of the future. 

Pro lect Staffing , Personnel Plan - 
ning , and Management 

Several difficulties in project 
implementation appear to have stem- 
med from problems of initial project 
staffing, personnel planning, and 
personnel management. 

Staffing and personnel planning . 
The success of most human service 
endeavors depends heavily on the 
staff involved in the del ivery of 
services . One difficulty in the 
staff ing for th is pr o j ect has 
already been described: The lack of 
educational credentials necessary to 
create an impression of legitimacy 
in the school. An additional diffi- 
culty may have stemmed from a mis- 
match between the job demands and 
personal preferences or talents of 
some staff. The artist-instructors 
were, we assume, for the most part 
individuals with interests and com- 
petencies in various artistic 
endeavors. Their experience and 
interest in the more conventional 
aspects of their jobs — developing 
curricula, keeping records, adhering 
to regular schedules, and so 
forth — may have been less than was 
required by the jobs . Our impres- 
sion is ilhit a more careful analysis 
of the tasks required by the jobs of 
proj ect director , instructor , and 
evaluation coordinator may have sug- 
gested the recruitment of a somewhat 
different mix of staff. This 
impression receives some support 
from the unsystematic reports we 
have received of job dissatisfaction 
on the part of some staff and from 
the relatively high turnover among 
project personnel. Both the project 
director and evaluation coordinator 
resigned, and turnover among staff 
overall exceeded 30%. 

Management . Midway through the 



project, project managers discovered 
a need for a structure for holding 
artist-instructors accountable for 
their activities in the schools. At 
that point they provided training 
for staff on the goals of the proj- 
ect, and with the .^hange in project 
directors they began providing more 
supervision. This increase in 
structured expectations , training , 
and supervision may have come too 
late in the life of the project. 

Effects of the Intervent'ion 

Despite the limitations in the 
evaluation described earlier, some 
^reasonably confident conclusions are 
possibl e. The fol lowing paragraphs 
describe our best summative judgment 
about the effects of the Jazzmobile 
Alternative Arts project. 

Empirical evidence . On the basis 
of the empirical evidence, the proj- 
ect had little effect on the stu- 
dents exposed to its interventions. 
Statistically signficant effects 
favoring treatment students were 
found on attachment to school (lik- 
ing for school) and on dropout. The 
size of these effects were moderate, 
ani confidence in their interpreta- 
tion as actual effects of the alter- 
native education project is weakened 
by the quasi-experimental nature of 
the design and the unavailability 
sufficient pre'-'intervention measures 
for thorough application of appro- 
priate statistical controls. .Fur- 
thermore, because fo many different 
outcomes were exam:-ned, one or more 
significant results were to be 
expected by chance alone. The proj- 
ect does not appear, however, to 
have done harm. The only statisti- 
cally significant result favoring 
the control group was on a measure 
of student employment: More control 
than treatment students reported 
full- or part-time employment. 
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The evaluation design was, 
despite its limitations, suffi- 
ciently strong that had the project 
produced large positive effects they 
should have been detected. We find 
no evidence of such effects in the 
empirical evidence. 

Artistic evaluation . Th e impl e- 
menting organization did not antici- 
pate an empirical evaluation of the 
kind we had hoped would be imple- 
mented (and of the kind that eventu- 
ally was implemented in substantial 
degree ) . Indeed , the organization 
indicated that it had hoped for a 
more artistic evaluation and that 
had it known about the kind .-of 
empirical evaluation expected it 
would not have undertaken the pro- 
ject in the first place. ^ 

The descriptions of artist-in- 
structor approaches to their work, 
included in an appendix to this 
report, are therefore of considera- 
ble interest. These descriptions 
suggest that artist-instructors were 
sensitive, caring, individuals. We 
have no way of knowing how accu- 
rately these descriptions character- 
ize the workaday acti'uities of these 
instructors, but we havfe no reason 
to doubt that they convey an accu- 
rate impression. An iviteresting 
impl ication of these descriptions , 
combined with the empirical evidence 
about proj ect effects , is either 
that (a) artist- ins true tors imple-" 
menting the kinds of personal philo- 
sophies and approaches described in 
the appendix is an insufficiently 



1 A letter from S. David Bailey 
(dated 30 April 1983) commenting on 
a draft of this report is available 
from Jazzmobile, 159 West 127th 
Street, New York, NY 10027, or from 
Lois Hybl , Center for Social Organi- 
zation of Schools, The Johns Hopkins 
University, 3505 N. Charles St. , 
Baltimore, bID 21218. 
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powerful intervention to produce 
much in th e way of me as ur abl e 
changes • in student behaviors and 
attitudes, or (b) these philosophies 
and approaches were not fully imple- 
mented on a day-to-day basis . The 
empirical evidence does not allow us 
to chose among these two alternative 
interpretations. 

Conclusion . /The al ternative . arts 
education intervention as imple- 
mented and studied here did not have 
a maj or influence on the psychoso- 
cial development of' youths in the 
community in which it operated.. It 
may have increased liking for 
school, prevented some dropout, and 
decreased student empl oyment , alt- 
'-'hough the evidence is not ironclad 
that these effects were produced by 
the intervention. No evidence sug- 
gests that this intervention pre- 
vented delinquent behavior. 

Recommendations for Future Projects 

The conduct and evaluation of 
this project have been instructive, 
and they suggest the fol lowing 
recommendations for future del in- 
quency prevention projects: 

1 . Grant awards for demonstra- 
tion projects should be made only to 
organizations that understand and 
agree to fully implement a rigorous 
evaluation, and where concrete evi- 
dence of a commitment of relevant 
actors in the organization^'s force- 
field to fully cooperate with the 
evaluation is available. 

2 . Expectations f pr evaluation 
should be made concrete and explicit 
in advance of requesting applica- 
tions for grant competitions. 

3. Demonstration projects 
involving only arts education show 
limited promise for del inquency pre- 
vention, and demonstration projects 
proposing only to implement arts 



177 



^« 1^ w^i^ IhAl dfti ^ducAKlon ftn 

li i«» ^ ^VidUf4i on))r ih^t oihpf 
thuiH ^if^n i»^ui} i mtp compUx 

Cit i>vit¥«lioii» Thp iKiblic dttidrvei 
pf0%fm% to prrvi^Rt ddlin^utncy 
^ith dr» dMMfifttfAbly «rrtetiye» 
Mi ^o|<»«t if^ii*ori »hoyU thore- 
toft Imi«» ^cUioni «)H>ut projoet 
tiiMiiiK im (*) itie«moDti of evalu- 
Atibilltyi m4 <b) th« mpirical and 
l(^#<Hr«ticAl Ut^raturt on dol in- 
i|iiH^«ir)r« fithi^r th^iH upon ivproiiioni 



about what leciina appeal ing« 

5* Prevent ion project implemen-* 
ten should carefully consider (and 
ahould be provided with assistance 
in considering) the potential obsta-* 
cles to project impiotnontation in 
the project^s environment. 

6 • Prevent ion pr o j ec t impl emen-^ 
ters should carefully consider (and 
should be provided vith assistance 
in determining) the personnel 
requirements of their interventions 
and in selecting and training the 
requisite personnel* 

7. Del inquency prevention pro-- 
,^Jects should initially and continu-* 
ally demonstrate 4 clear and direct 
commitment to the r^evention of del-^ 
inquency as a primary goal. 
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Figure 2 
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Figure 3. Organizational Arrangements 
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Table 1 



Jazzmobile Alternative Arts P^oji^cts: Goals 
and Corresponding Measures for V/hich Data 
Were Actually Provi^Qd 



Goal 



Measure 



1, Reduce truancy and chronic 
absenteeism 



a. Data on absences provided 
by the school 

b. SAES Student Questionnaire 
Non-attendance Inde^c 



2, Reduce youth involvement 
in "street" activity which 
promotes delinquent behavior 



None 



3. Youth potential will be 
used in a positive way. 



None 



4, Increased parent awareness 
and involvement 



None 
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Jazzmobile Alternative Aft^ Project: Obj^^tives 
(i.e. Intennediary Outconii^s) and Correspot^dix;g 

Mea^ut^s 



Objective 



Measure 



1. Increased student-teacher 
attachments 



Student-teacb^>^ 
Interaction^ 



2. Studentc will become 
excited and stimulated by in- 
school experiences 

3. Increase youths' career 
awareness, primarily in the 
area of art, Secondarily in 
other areas. 



Attachment to 
school^. 



yone 



4 . Increase students' success 
in non-acadetnic areas (i.e. the 
arts ) 



None 



5. Increase parental awareness 
concerning school policies, 
procedures and practices with 
regard to discipXinary actions 
and school referral s to the 
juvenile justice system 

6 . Increase parents' concern 
and support for youths' school 
activities 

?• Youths previously unused 
potentials will be tapped 

8. Increased youth value of 
education 

9. Reduce youths' sense of 
Alienation^ 



None 



Parental EmpJ^^^is on 
Education sc^l^^ 



None 



Educational ira^ions'^ 



Alienation sc^^^^ 



^The project Personnel did not identify these t^i^^Sutr^s; 

they seem to Us to be relevant and were availat?!^ tJi^J^^ugh the 

SAES questioJin^ire. 
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Table 3 

Educatic>^al Background of Jazzmobile 
Alternative Arts Education Project 
ae of June, 1982 



Wof^ locci^^on and eciucatioi^ FTE Number 



JazzmotUe office^ 7.00 9 

M.A- 2 

B.A^/g.S. 4 

A. A- ) 1 

Schools • 8.11 15 

M,A- 2 

B. A-/B.S. 4 
A.A» 1 

High school 8 

Total 15.11 24 



Ko_te> Of ^be 35 people ever employed by the-project, 11 (31%) 
had left ^i^c project's staff by June, 1982 (Source: SAES quar- 
terly staffing report), 

^Includes project in^^sger> project director, curriculum spe- 
cialistj ^ccountantj audiovisual specialist, field worker, and 
secreta^^gs. 
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Table 4 

Jazzmobile Alternative Education Arts Project: 
Means and Standard Deviations for Spring 1981 
Student Survey Outcomes for Students in Treatment 
and Non-Equival ent Control Groups 



Group 











Non- 


•equiv al ent 




Treatment 






Control 




Variable 


M 


SD 


N 


M 


•SD 


N 


Attachment to Parents 








a 


^ 


Negative Peer Infltience 


-.34 


4.40 


1 1 Q 


-.35 


4.92 


77 


Attachment to School 


-.50 , 5.88 


'i 1 


-.47 


6.66 


39 


Negative Belief 


2.68 '^1.32 


20 


2.05 


1.13 


34 


Interpersonal Competency 


4.00 


1.29 


1 o 

19 


3.99 


.90 


35 


Positi\ 3 Self-Concept 


-.60 


4.39 


18 


.85 


5.28 


32 


Punishment Index 


-.75 


2.02 


43 


-.08 


2.54 


52 


Rewards Index 


.14 


3.69 


41 


.37 


3.07 


48 


Victimization 


.95 


1.48 


37 


.53 


.82 


50 


School Effort 


2.05 


.76 


162 


2.00 


.73 


95 


Practical Knowledge 


9.17 


5.78 


20 


9.68 


5.47 


40 


Internal -External Control 


2.80 


.95 


20 


3.06 


.76 


35 


Al lenation 


1.24 


1.24 


LL 


1.22 


1.43 


34 


Self-Repor tea Grades 


3.01 


.83 


162 


3.11 


.79 


94 


Reading Ability Self- 


1.39 


.88 


145 


1.68 


.80 


78 


Reported 














Satisfaction with School 


2.24 


.78 


167 


2.16 


.72 


95 


Progress 














School Nona t tendance 


1.04 


1.65 


166 


1.11 


1.72 


88 


Index 














Rebellious Autonomy 


2.05 


1.10 


20 


2.29 


.92 


36 


Involvement 


.09 


6.58 


102 


-1.06 


5.69 


69 


Total Del inquency 


1.88 


2.55 


78 


1.75 


2.54 


57 


Serious Del inquency 


.74 


1.49 


81 


.44 


1.33 


62 


Drug Use 


.63 


.98 


81 


.76 


1.17 


60 


Note. None of the outcomes 


presented 


above 


show 


statistically signi- 


ficant differences between 


treatment 


and control 


students. 







^The questions comprising this scale did not appear on the Jazzmobile 
version of the 1981 Student Survey due to objections by the Community 
School Board. 
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Table 5 



Jazzmobile Alternative Arts Project: Means 
and Standard Deviations for Spring, 1982 
Student Survey Outcomes for Students in 

Treatment and Non-Equivalent Control Groups 



Group 



Non-equival ent 
Treatment Control s 



Variable 


M 


SD 


N 


M 


'SD 


N 


Attachment to Parents 


• 63 


• 26 


O 1 

81 


.60 


.25 


42 


Negative Peer influence 


• 16 


• 19 


75 


.16 


.15 


39 


Attachment to School 


• 81* \ 


> • 18 


72 


.72 


.23 


39 


Belief Scale Scored Positively 


•7 1 

• 71 


• 22 




• 75 


• 24 


33 


Interpersonal Competency 


o c 

• 85 


1 o 

• 18 


45 


• 76 


• 26 


32 


Positive Sel f-Concept 


• oU 


1 A 
• 1^- 


46 


(J A 


1 o 

• 18 


J 1 


Punishment Index 


• 20 


.22 


77 


1 o 

• 18 


• 26 


o o 

38 


Rewards Index 


.37 


.32 


77 


• j6 


.31 


o o 

38 


Victimization 


f\n 

• uy 


. 1 J 


77 


• 1 J 


.20 


38 


School Effort 


.67 


.26 . 


.80. - 


^ .59. . 


.26 _ 


.38 


Practical Knowledge 


1.31 


.34 


38 


1^39 


.29 


30 


Internal -External Control 


.44 


.21 


48 


• 41 


.24 


31 


Alienation 


.27 


.22 


48 


• 30 


.30 


33 


C A 1 r Qr\/'\'V a ^ fZ •** «3 ^ Q e? 

D6ir IxcpOLLcQ vyraocs 


3.18 


.67 


oo 




.92 


42 


Reading Ability, Self Rating 


1.56 


.65 


79 


1.74 


.92 


38 


Days of School Cut in 














4 Weeks Preceding Survey 


.10 


.30 


89 


.21 


.68 


42 


How Often Cut One or 














More Classes 


.39 


.73 


89 


.25 


.62 


44 


Educational Expectation 


3.99 


1.35 


87 


3.53 


1.80 


45 


Did You Work for Pay Last 














Week? 


.46 


.50 


86 


.65 


.48 


40 


Regular Part-Time or 














Full-Time Job? 


.11* 


.31 


84 


.30 


.56 


43 


Suspended from School During 














Term 


.16 


.37 


69 


.18 


.39 


38 


School Nona ttendance Index 


.38 


.58 


87 


.31 


.64 


42 


Rebellious Autonomy 


.60 


.34 


37 


.52 


.36 


30 


Involvement 


.24 


.18 


77 


.24 


.20 


37 


*Treatment group significantly 
(p<.05). 


different from 


non- 


•equivalent 


controls 
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Table 6 

Jazzmobile Alternative Arts Project: Means 
and Standard Deviations for 1981-82 
Outcomes Measured Using School Records 



Group 



Non-equivalent 
Treatment controls 
Outcome ~ 



var labl e 


M 


SD 


XT 


M 


SD N 


April, iyo2, California 












Achievement Test (CAT) 












Grade Equivalent Scores 












Reading 


7.69 


1.99 


105 


7.53 


2.93 51 


Math 


7.32 , 


1, 1.55 


102 


6.88 


1.99 48 


Number of days absent 


15.11 


16.80 


106 


13.82 


18.46 56 


during the school 












year^ 












Promoted to next grade 


.93 


.26 


111 


.81 


.40 62 


( l=promoted, O=not )^ 












Withdrawal for reasons 


.05* 


. .21 


195 


.13 


.34 156 


other than transfer^ 












Note. N's vary depending 


on the availability 


of 


data. Data on with- 


drawals were provided for 


the entire 


sample. 


Parental permission was 


required for the release 


of other information 


. to 


the evaluation. Per- 


mission was obtained for 


a total of 183 students. 


Attendance, promo- 


tion, and test score data 


were missing for a 


few 


of these 


students. 


a • 

No data were available 


for students 


who withdrew during 


the school 



year. 

^ The unadjusted proportion of students promoted was significant at 
the p<.05 level. The treatment and control group also differed at the 
.01 level on prior promotions. An analysis of covariance, treating 
prior promotions and special education status as covariates, implies 
no significant difference (p=.78) between treatment and control 
groups. Because prior promotion data were unavailable for many stu- 
dents, the N's for this analysis were 76 and 18. 

c „ , , 

No pre-treatment data were available for students who withdrew. 

Therefore we were unable to perform analyses that control for pre- 
treatment differences. 
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Appendix A - Job Descriptions Provided by Jazzmobile 



Executive Director/Program ManA >' ::er 

Manages program to insure that implementation and prescribed activities are 
carried out in accordance with specified objectives and in accordance with 
O.J. J. D. P. guidelines: Plans and develops methods and procedures for imple- 
menting program, direct s and coordinates program activities, and exercises 
control over personnel responsible for specific functions or phases of pro- 
gram. Sel ects personnel according to knowledge and experience in area with 
which program is concerned, arts education. Confers with staff to explain 
program and individual responsibilities for functions and phases of program. 
Directs and coordinates through subordinate managerial personnel , activities 
concerned with implementation and carrying out objectives of program. Reviews 
reports and records" of activities to insure progress is being accomplished 
toward specified program obj ect ives and mpdif ies or changes methodology as 
required to redirect activities and attain objectives. Prepares program 
reports for the Office of Juvenile Justice and Delinquency Prevention. Con- 
trols expenditures in accordance with budget allocations. Meets with the 
Advisory-Committee and District Representatives. Reports to the President and 
Board of Directors of Jazzmobile, Inc. 



Educational Specialist 

No description available. 



Proiect Director 



Under the direct supervision of the Program Manager, plans, directs, and 
coordinates activities of designated project to insure that aims, goals, or 
objectives specified for project are accomplished in accordance with pre- 
scribed priorities, time limitation, and funding conditions: Reviews project 
proposal or plan to ascertain time frame and funding limitations, and to de- 
termine methods and procedures for accomplishment of project and staffing 
requirements. Establishes work plan and schedules for each phase of project 
in accordance with time limitations and funding. Confers with staff to out- 
line project plans. Modifies schedules, as required. Prepares project status 
reports for management. Confers with project personnel to provide technical 
advice and to assist in solving problems. Oversees daily project operations 
in all three school sites. 

Art is t/ Instructors 



Art. Instructs pupils in art, such as painting, sketching, designing, and 
sculpturing : Demonstrates method and procedure to pupils. Observes pupils' 
work to make criticisms and corrections. Directs planning and supervising of 
student contests and arranging of art exhibits. Conducts workshops for 
parents regarding the art discipline the student is learning. 

Drama . Teaches acting principles and techniques to groups: Conducts read- 
ings to evaluate student^'s talent. Adapts course of study and training meth- 
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ods to meet student's need and ability. Teaches enunciation, diction, voice 
development, and dialects, using voice exercises, speech drills, explanation, 
lectures, and improvisation. Discusses and demonstrates vocal and body 
expression to teach acting styles, character development, and personality pro- 
jection. Directs plays for school and public performances. Auditions stu- 
dents to select cast and assign parts. Rehearses and drills students to 
insure they master parts. Assigns nonperf orming students to backstage produc- 
tion tasks, such as constructing, painting, moving scenery, operating stage 
light, and sound equipment. Directs activities of students involved in con- 
structing, painting, and lighting scenery. Teaches elements of stagecraft, 
stage makeup, costume craft, play writing, and play direction. Conducts work- 
shops for parents of participating students regarding the art discipline the 
student is learning. 

Dance . Instructs pupils in bal let , modern and ethnic forms of dancing: 
Observes students to determine physical and artistic qualifications and limi- 
tations and plans programs to meet students' needs and aspirations. Explains 
and demonstrates techniques and methods of regulating movements of body to 
musical or rhythmic accompaniment. Drills pupils in execution of dance steps. 
Teaches history of dance. Choreographs and directs dance performance. As 
instructor, conducts workshops for parents of participating students regarding 
the art discipline the student is learning. 

Guitar . Teaches groups guitar in public school: Plans daily classroom 
work based on teaching outline he/she has prepared for this special program. 
Evaluates student's interests, aptitudes, temperament, and individual charac- 
teristics to determine abilities. Instructs students in music theory, har- 
mony, score and sight reading, composition, music appreciation, and provides 
group lessons using technical knowledge, aesthetic appreciation, and appropri- 
ate teaching techniques. Conducts workshops for parents of participating stu- 
dents regarding the art discipline the student is learning. 

Percussion . Teaches groups percussion in publ ic school : PI ans daily 
classroom work based on teaching outline he/she has prepared for this special 
program. Evaluates student's interests, aptitudes, temperament, and indivi- 
dual characteristics to determine abilities. Instructs students in music 
theory, harmony, score and sight reading, composition, music appreciation, and 
provides group lessons using technical knowledge, aesthetic appreciation, and 
appropriate teaching techniques. Conducts worksbpps for parents of partici- 
pating students regarding the art discipline the student is learning. 

Evaluation Coordinator 

Directs research acitivities concerned with the program: Initiates proce- 
dures to determine if program objectives are being met. Provides interpreta- 
tion of research data gathered. Works as the liaison between the program and 
the national evaluators concerning the management information system. Pro- 
vides evaluators with the data as outlined in the evaluation design. Coordi- 
nates data collection methods with the project director and school staff. 
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Counselor 



Counsels individuals and provides group educational and vocational guidance 
services relating to the arts: Collects, organizes, and analyzes information 
about individuals through records, tests, interviews, and professional 
sources, to appraise their interests, aptitudes, abilities, and personality 
characteristics, for vocational and educational planning. Compiles and stud- 
ies occupational , educational , and economic information to aid counsel ees in 
makir.g and carrying out vocational and educational objectives. Refers stu- 
dents to placement service. Assists individuals to understand and overcome 
social and emotional problems. Assists gifted and talented students with 
placement into special arts high schools. Provides arts activities and educa- 
tional opportunities information for parents of the participating students. 

Field Worker 

Works s the direct liaison between the program and the community. Under 
the direction of the Project Director organizes and conduct s workshops for 
parents of the participating students regarding the program goals, objectives 
and their involvement. Also conducts parent workshops in conjunction with the 
guidance counselor and artist/instructors concerning their subject areas. 
Assists the Guidance Counselor with disciplinary problems requiring parental 
involvement. 

Curriculum Coordinator . 

Plans and develops alternative education curriculum materials integrating 
the arts into basic subject areas for grades 6, 7, and 8: Develops materials 
for in-service education of teaching personnel. Confers with local school 
officials to develop curriculum materials. Confers with lay and professional 
groups to disseminate and receive input on curriculum materials. Conducts 
conferences designed to promote arts education activities related to core cur- 
riculum areas. Conducts research into areas, such as teaching methods and 
techniques. 

Accountant 

Manages all fiscal activities for the O.J.J.D.P. program. Applies princi- 
ples of accounting to devise and implement accounting system. Maintains 
accounts and records. Performs such bookkeeping activities as recording dis- 
bursements, expenses, and tax payments. Prepares quarterly fiscal reports. 
Prepares cash advancement requests. Prepares weekly payroll. Represents 
Jazzmobile before government agency upon certification by agency involved. 

Audiovisual Production Specialist 

Plans and produces audio, visual , and audiovisual material for communica- 
tion, information, training, and learning purposes: Develops production ideas 
based on assignment and generates own ideas based on objectives and personal 
interest. Conducts research or utilizes knowledge and training to determine 
format, approach, content, level, and media which will be -nost effective, meet 
objectives, and remain within budget restrictions. Plans and develops prepro- 
duction ideas into outlines and scripts. Locates and secures settings, prop- 
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erties, effects, and other production necessities. Sets up, adjusts and 
operates equipment, such as cameras, sound- mixers, and recorders during pro- 
duction* Conducts training sessions on selection, use, and design of audiovi- 
sual materials and operation of presentation equipment. 

Secretary 

Schedules appointments, gives information to callers, takes dictation, and 
otherwise relieves officials of clerical work and minor administrative and 
business detail: Reads and routes incoming mail. Locates and attaches appro- 
priate file to correspondence to be answered by employer. Types routine cor- 
respondence. Files correspondence and other records. Answers telephone and 
gives information to callers or routes call to appropriate official and places 
out going calls. Schedules appointments for employer. Greets" visitors, 
ascertains nature of business, and conducts visitors to employer or appropri- 
ate person. Arranges travel schedule and reservations. Compiles and types 
statistical reports. Makes copies of correspondence or other printed matter, 
using copying machine. May prepare outg9ing mail , using postage-metering 
machine. Types all curriculum materials. 

Clerk/Typist 

Compiles data and operates typewriter in performance of routine clerical 
duties to maintain evaluation data: Types reports, correspondence, applica- 
tion forms, and data for evaluation. Files records and reports, posts infor- 
mation to records, sorts and distributes mail, answers telephone, and performs 
similar duties. 
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The following promotional material, prepared by Jazzmobile staffer V/illiam 
Harris, presents their artist instructors in a positive light. These "artis- 
tic evaluations" are interest:.ng and serve to give some of the flavor of the 
project's activities. We present them without further comment. 
Wendell Williams 

Wendell Williams, a guitar instructor, relates his first experience of per- 
sonal counseling of one of his Jazzmobile students. A 7th grade student ten- 
tatively indicated that he smoked marijuana. He later confessed that all of 
the money from his after-school, part-time job was going toward purchasing the 
drug which he realized had become "a problem." He did not feel that he could 
talk to his parents about it, and fortunately Mr. Williams was an understand- 
ing, but nonjudgmental adult with whom he could discuss his "dilemma. ^ - 

Williams emphasized that although he was sympathetic, concerned and suppor-- 
tive, the solution had to come from the student himself. The student was able 
to overcome his problem and thanked his instructor for his help and concern. 
This incident taught V/illiams the importance of open communication with the 
members of his class. He instituted loosely scheduled class periods for gen- 
eral discussions of their problems and goals. Sometimes one or more of the 
students would feel the need for thrashing out problems and would initiate the 
talk sessions. 

"The whole thing is getting them on your side," Williams says. During 
these sessions, in addition to their problems, he discusses the positives and 
negatives of the life of an aspiring artist, and also explores career alterna- 
tives which most students had little knowledge of. 

Earl Stewart 

Percussion instructor. Earl Stewart, had his first contact with a neighbor- 
hood family when one of their six sons entered his class. Although he was one 
of Stewart's best students, he was eventually lost to the streets. 

The three younger brothers, 12, 10, and 8, all good students, are now tak- 
ing percussion. Stewart realizes that the street life and the "quick and 
easy" money are as great a lure to them as it was to their brother. He also 
realizes that without the positive attraction of Jazzmobile, it is a real pos- 
sibility that they would already have fallen into "the life. 
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His method of combating the reality of that negative attraction is to 
constantly remind them of the alternative. Jazzmobile, he points out, is more 
than a day camp. To prove his point, he now has them involved in the project 
to raise money through performances. 

Self-discipline is constantly stressed as a necessary ingredient for 
remaining in his clsss, and for making positive progress in their own lives. 

Stev/art has appointed the oldest of the brothers as his drum captain. He 
has taken the students to the rehearsals of his professional group. Although 
the final verdict is far from being in, Stewart believes the brothers are 
beginning to realize that they do not have to resort to the life style of 
their older sibling. 

Elba Mulev 

Ms. Muley had an art student 12 or 13 years old. She found that he had a 
history of physical run-ins with his instructors and had spent time in a dis- 
ciplinary school situation as a result. 

His abilities with the visual arts, expressed primarily through wall graf- 
fiti, were obvious, but sports and physical aggressiveness held his interest 
more than art. 

Muley developed a project for a few of her students in various classes to 
create comic books which 5poke to the problem of violence, stealing and drugs 
as they affected the school environment. The student, working independently 
by his own choice, developed the storyline, characters, dialogue and art work 
for the book. He worked for many weeks on the project. Though his interest 
was evident, it was a constant struggle to motivate him on a daily basis. 

When the work was nearly completed he found out that others were also 
involved in similar pursuits. This angered him to the point that he tore up 
the materials that he had worked so long and hard on. Ms. Muley asked him to 
leave the class. He did, but returned a few days later. They discussed their 
misunderstanding both as it related to the nature of the project, and their 
personal conflict which resulted from that misunderstanding. 

With this new understanding the student set about recreating his manu- 
script. He used his lunch periods and any free time he had to meet the dead- 
line for completing his submission. Tl\e student's aunt contacted his instruc- 
tor to commend her for the marked change evident in his behavior. 

The book was finished on time and was one of only three to receive an award 
in an art competition. He later completed a second book about graffiti. 

Miriam Bacote 

A girl, who Ms. Bacote soon realized lacked self confidence, entered her 
dance class. She came as a member of a clique of girls who thought the dance 
class was to simply involve learning the laitest popular steps. When they 
found what was actually expected of them they withdrew from the class. The 
girl stayed, but was torn by an on-going conflict between her desire to follow 
her friends, and her interest in learning to dance. 
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Ms. Bacote contacted the student's mother in the hope that encouragement 
from home would help to relieve the girl's anxiety about her choice. Unfortu- 
nately, the mother was more concerned with her own street activities and was 
no help. The student obviously benefited from Bacote's concern, and came to 
her own understanding of the value of the discipline involved in an endeavor 
that she loved. She developed to the point that she was featured as the lead 
dancer in one of the class's performances. 

Roberto Reves 

Guitar instructor Roberto Reyes is well ax^are of the problem of providing 
technical instruction to students with widely varied levels of aptitude for 
the instrument. Using a method of initially having them copy the things that 
he plays, he begins to get a feel for their coordination and ability to memor- 
ize, as well as getting them all moving at the same time at their various lev- 
els. Step by step encouragement is most important at this stage, giving 
credit for even the smallest accomplishment. Actual musical notation comes 
only after everyone has a newly acquired sense of the confidence that they 
have some facility on the instrument. '» 

Reyes' method of teaching his students to respect their instrument paral- 
lels his philosophy of dealing with them as human beings. The period leading 
to the actual hands-on experience with the guitar is highly structured. The 
question of who paid for the guitars is discussed. Once it is understood that 
their parents are, in essence, owners of the instruments, a sense or responsi- 
bility and privilege is connected with their use. - The students know that any 
abuse of the instrument means revocation of the privilegeof using it. He 
emphasizes that he has never lost a guitar through abuse or theft. 

Felice Batiste 

The image this art instructor tries to project is that he is his students' 
friend as well as their instructor. In addition to the specific techniques of 
artistic expression, Mr. Batiste strives to instill in them an appreciation 
of art and its varied experiences. 

He maintains an "open door policy," always providing students with a 
friendly, impartial ear. He attempts to explain the seemingly unfair contrad- 
ictions they are faced with. Batiste, who does not insist that his students 
call him by his surname, says that he does not ever remember having a free 
lunch period; there always seem to be a number of students who want to talk. 

Batiste initially concentrates on classroom assignments which can be com- 
pleted in one session; both as an alternative to the ongoing pressures of the 
rest of their school experience, and as an attempt to instill a steadily grow- 
ing sense of accomplishment and self-confidence. 

An example of his one-on-one contact with his students is a situation that 
occurred in a previous summer's program. Batiste had an introverted, but 
talented student who should have been graduating but had been left back. She 
wanted to go to an art school but was intimidated by the art test which was a 
prerequisite to entrance in the school of her choice. She was discouraged to 
the point of wanting to quit school. Batiste helped her with the basics of 
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the school test and convinced her of the importance of staying in school, 
eventually passed the exam, and is now doing well. 
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Otro Camino: Second Interim Report 



J. St. John 



vOtro Camino is a community-based program that functions as an exten- 
tion of the school day. Project staff work cooperatively with local 
school administrators and faculty to assess student academic needs and 
schedule program activities. Besides the academic component, the pro- 
gram includes activities aimed at student social development and commu- 
nity participation in services offered. 

Interim results suggest that the program may be increasing Otro Cam- 
ino participants' school grades, performance on standardized tests, and 
educational expectations . Students' reports of influence in the school 
and involvement in extracurricular activities may al so be increased by 
the program. Negative findings for Otro Camino's academic service com*" 
ponent on arrest and reports of punishing experiences in school are most 
likely due to selection bias rather than to the program. 



Major Program Design Changes 

Changes to Community Development 
Components 

Advisory council . Otro Camino 
had planned to institute an advisory 
council during the first program 
year. Al th ough th e co unc i 1 wa s 
formed, the mission of the organiza- 
tion was not clear, nor was the com- 
position of membership specified. 
The council met only once that year. 
In the second year, project managers 
decided the council should have four 
committees: ^ ^ (a) educators, 
(b) community members, (c) students, 
and (d) an executive committee com- 
posed of el ected officers from each 
committee and project management. 
The educators' committee was to 
include the three school directors 
(principals), the district superin- 
tendent , each school ' s f acul ty 1 iai- 
son to Otro Camino, regional univer- 
sity f acul ty , and others , up to 1 1 



This report supplements an earlier 
repor t on this proj ect (St. John, 
1982) which should be consulted for 
an account of the project's first 
year of operation. 



members. Community members were to 
be representative of many aspects of 
communi ty 1 if e , incl uding bu sine ss , 
social services, police, and parents 
from several barrios in the la Playa 
community, with the majority on this 
11-person committee being parents. 
The student committee was to include 
a representative of each grade level 
served C altogether), the president 
of the Presidents' Club at Otro Cam- 
ino, student council representatives 
from the school , and other students 
who were interested in being on the 
council (up to 11 members total )• 

The council was charged by pro- 
ject managers to (a) assess whether 
the practices at Otro Camino were 
educationally sound and wo?i'k to 
improve the local schools (educa- 
tors), (b) assist Otro Camino in 
meeting the needs of the community 
(community members), and (c) assess 
whether or not Otro Camino student 
activities were adequate and suffi- 
cient (students). Their charge went 
beyond making recommendations and 
included assistance in impletfienting 
recommendations. 

The committees met separately, 
usually on the same evening, and 
after the separate meetings , the 
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total group met to hear the recom- 
mendations of each committee. The 
council did not begin functioning 
fully until January and reportedly 
met two more times that school year. 

Project staff reported on the 
recommendations made by the parents' 
commit tee dur ing the year. Commit- 
tee members were discouraged by the 
low turnout for their meetings and 
suggested that Otro Camino sponsor 
special community events to gain 
more visibility in the grass-roots 
community . They suggested that stu- 
dents nominate their mothers for 
receipt of . the "Mpther of la Playa 
Award" that would be given by Otro 
Camino. All nominees would be noti- 
fied and inviteci to the awards cere- 
mony at Otro Camino and thereby gain 
more information about the program 
and their children's participation. 
Otro Camino staff and students 
implemented this recommendation in 
the spring. The community committee 
recommendation resulted in the staff 
and students sponsoring several 
other special community events as 
well. Otro Camino sponsored obser- 
vations of International Women's 
Week, Holy Week, and Library Week, 
and hosted numerous celebrations of 
student participant achievements. 

School f acul ty support . SAES 
Spring 1981 student questionnaire 
responses indicated that students at 
all three schools were reporting low 
self-concepts and that their ability 
to interact with others was low (two 
schools were below the fifteenth 
percentile on both measures and the 
other school was below the thirtieth 
percentile on the SAES Self-Concept 
and Interpersonal Competency mea- 
sures). Otro Camino managers had 
designed their program to help stu- 
dents with their self-esteem and 
interactive skills. They predicted 
that as students improved in these 
areas they would feel more comforta- 
ble in the school and become more 



attached to school. But they 
regarded current teaching practices 
at the school s as an obstacle to 
their effectiveness. They hypothes- 
ized that attachment to school would 
increase only if teach ing practices 
in the regular school improved. 
Furthermore, they thought that per- 
ceptions of the regular school might 
even worsen for students who parti- 
cipated in Otro Camino activities. 
The program managers decided to 
institute teacher training courses 
for the target schools', faculties in 
hopes that some teaching methods 
would 'change. 

School administrators agreed to 
"turn over three half-day in-service 
training sessions each semester to 
Otro Camino personnel. Otro Camino 
offered workshops at two in-service 
sessions in the f al 1 semester, but 
later abandoned the practice. They 
felt that the competition between 
the school - organizations and -Otro 
Camino was being heightened by their 
efforts, and that the competition 
was hindering program operations. 

In the second semester, managers 
attempted to implement regular Otro 
Camino staff follow-up of partici- 
pants by contacting school f acul ty . 
This was to serve the purpose of 
letting teachers know that goals 
regarding students were shared, and 
that Otro Camino' s purpose was to 
support the efforts of the school s 
despite their different methods. 
However, very little contact between 
Otro Camino staff and school faculty 
actually occurred. Otro Camino man- 
agers reported that their efforts 
were often thwarted because school 
faculty schedules did not allow time 
to meet. 

Parent involvement . Project 
s t af f pi anned to invol ve many 
parents in the service s offered to 
students. Early attempts to solicit 
parent volunteers to assist with the 
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student service component met vith 
only limited success. Other strate- 
gies were developed to foster parent 
support of students. A staff com- 
mittee went to the advisory council 
and asked for suggestions on work- 
shop topics that staff could take to 
parents at community centers , 
recreation fields , government hous- 
ing project community halls, and the 
like. A schedule set the beginning 
of the workshops in the second sem- 
ester of the 82-83 school year. 
Staff planned for the public rela- 
tions club to help with publicity by 
broadcasting workshop times and 
places by loud speakers from cars. 
The staff audiovisual specialist was 
to arrange for electrical power at 
outdoor sites so that slides, video 
t ape s , and mov ies could be par t of 
all workshops. At each workshop, 
staff planned to survey parents to 
find out what kind of education top- 
ics they would like to have other 
workshops on, and these topics would 
be added to the list for other 
times. This strategy was thought to 
be responsive to the perceived needs 
of parents. Implementation data for 
the 1982-83 year will show whether 
or not the strategy was successful 
at increasing parental participa- 
tion. 

Changes to Student Services Compo- 
nents 

Academic assistance at Otro Cam - 
ino . Academic assistance has been a 
major component of Otro Camino since 
its inception. In the first year 
the component took the form of indi- 
vidual tutoring in subj ect areas in 
which students reported having dif- 
ficulty. This approach was used 
because the program was a drop-in 
center and students attended when 
they had time. In summer, 1981, 
students participated for the full 
day. To accommodate the schedule 
change, the project adopted a small 
group instruction model. Project 



staff and managers believed this 
model was effective and implemented 
small group instruction for the 
81-82 school year on a semester 
basis. Project managers bel ieved 
that the small group instruction 
approach would make it possible to 
serve larger numbers of students. 

Three distinctly different kinds 
of academic assistance were offered 
using the smal 1 group approach. 
During the first semester , groups 
were involved in creative learning 
activ it ies to reinforce specific 
acadeiriic skills. These activities 
were intended to be fun. For ins- 
tance , some students who reported 
\» they needed help with Spanish read- 
ing skills selected placement in a 
group that read and discussed the 
daily newspaper. During the second 
semester, groups were conducted 
using more traditional materials and 
traditional tutoring methods. That 
is, students brought in their school 
books and Otro Camino staff helped 
students complete assignments. Stu- 
dents were also tested for areas of 
weakness and given supplemental 
materials and instruction in these 
areas. 

School administrators requested 
that Otro Camino provide intensive 
tutoring to a group of students 
identified as potential dropouts. 
The schools sent 35 students in the 
seventh and eighth grades to the 
program site beginning in early 
March and ending in mid-May (the 
close of the school year) . Otro 
Camino staff divided the group into 
five small groups. An elaborate 
schedule was established that pro- 
vided two 50-minute periods of 
tutoring each school morning. The 
daily schedule included rotation 
through two subjects and allocated 
four 50-minute periodic to six sub- 
jects in a ten-day period. 
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Library science * Otro Camino 
managers arranged to train five high 
school students in library proce- 
dures and techniques for high school 
credit. These students worked in 
Otro Camino's 1 ibr ary learning book 
cataloging methods, operating a book 
lending system, and assisting with 
locating references for students. 
Four hours a week were spent working 
in the library, and students 
received one school credit for the 
course. 

Otro Camino mini-courses at the 
schools . Each Otro Camino staff 
member developed a mini-course dur- 
ing the first semester for possible 
implementation at the schools in the 
second semester. Staff contacted 
school teachers to ask permission to 
incorporate the mini-course into the 
regular classroom courses. Mini- 
courses were designed to last ten 
days, had a limited number of learn- 
ing objectives, included daily les- 
son plans, and were to be given by 
Otro Camino staff with the teacher 
present in the room. The courses 
covered special topics in Spanish, 
English, math, science, social sci- 
ence , and folk crafts. Each course 
was devel oped for a par ticul ar age 
group. Not all teachers agreed to 
participate. Project managers 
reported that three of these courses 
were given during the 81-82 school 
year, and that their course on the 
la Playa community was being given 
in each tenth grade social science 
class beginning in August, 1982. 

Clubs . Otro Camino attempted to 
promote student social development 
through clubs. Otro Camino devel- 
oped and implemented 19 clubs during 
the 81-82 school year. Project 
plans had called for the implementa- 
t ion of f ive cl ubs beg inning in the 
first program year , but none were 
begun until the second year. The 
implemented clubs covered personal 
growth (career and psychological ) , 
recreation, arts and crafts. 



investigative science, and drama and 
performing arts. Two clubs were 
organized to address educational 
issues . in the community, and stu- 
dents organized a publ ic rel ations 
club to help promote Otro Camino. 
Executives from each club composed a 
club of presidents who coordinated 
activities and kept all participants 
and staff up to date on each club. 

One club had a special purpose. 
Several youths with discipline prob- 
lems were referred to, the project 
from the high school . Repeated 
counseling efforts had proven unsuc- 
cessful except in el iciting some 
personal goal s. Several students in 
'» this group indicated they wanted to 
become martial arts experts. A 
r el a t ive of an Otro Cam ino staff 
person owned a karate school and 
volunteered to donate one class per 
week day for these students. In the 
class, students learned the discip- 
line necessary to -become —experts, 
and school personnel reported the 
students' self-discipline improved 
markedly at school . Once the stu- 
dents had mastered basic karate 
maneuvers, they performed at a 
school assembly, and appeared to 
have become respected members of the 
s chool communi ty . I observed a 
well-received presentation of skills 
given by karate club members during 
Otro Camino's week celebrating stu- 
dent achievements in April. 

Learning resource center . Origi- 
nally Otro Camino planned for the 
learning resource center to be used 
by drop- in students (students who 
were not registered in formal activ- 
ities) as an additional resource for 
their school assignments. The cen- 
ter included a library and audiovi- 
sual center. In the second year of 
operation, very few non-registered 
students used the center, but large 
numbers of registered students used 
the center facilities at times they 
were not scheduled for activities. 
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Ma^r Force Field Changes 

A oev school opened in August of 
1981 and caused unanticipated delays 
in Otro Camino activities. Two tar- 
get schools changed from double-ses*- 
siott to single-session days and re- 
distributed grade levels. Resulting 
school enrollment and scheduling 
problems lasted for about a month* 
Otro Canino managers had to wait 
until school schedule problems were 
resolved to establish their sche- 
dule. 

When Otro Camino finally began to 
schedule students into activities, 
project managers discovered that 
their plan to serve students during 
the school day could not be imple- 
mented* The schools had added elec- 
tives to the school curriculum and 
students no longer had free periods 
during which they could participate. 
Project managers quickly assessed 
the feasibility of after-school and 
week end activities and received a 
positive response. The ensuing 
reorganization further delayed pro- 
ject start-up» 

Changes in schedules meant that 
large numbers of students were 
served in a few hours each day. 
This necessitated offering courses 
that met two or three days a week at 
alternate times rather than every 
day because of space shortages* 
Project managers also gained school 
administrators'' permission to 
arrange with school teachers for 
Inclusion of ten-day mini-courses 
given by Otro Camino staff in regu- 
lar school classrooms* 

Some changes were salutory. Otro 
Camino'' s good reputation spread 
through the community and resulted 
in the enlistment of several commu- 
nity volunteers* A parent volun- 
teered to help with tutoring i area 
university professors assisted in 
diagnosing students'' academic needs 
and designing treatment programs > a 



local artist offered a class in folk 
art, 15 comnu^nity businesses opened 
48 job training slots for partici- 
pants, the local school of karate 
donated one class each day to a 
smal 1 group of participants, the 
Ponce Regional Office of the Puerto 
Rico Department of Education sent 
personnel to teach community inter- 
view and questionnaire design and 
statistics to participants, and the 
police department sent a representa- 
tive to the advisory council. The 
Puerto Rico Department .of Education 
Research and Evaluation office coop- 
erated in sending standardized test 
scores to the national evaluation 
project • 

Implementation Study 

Project staff kept careful 
records on student attendance at 
Otro Camino by activity attended. 
At the end of each semester the 
daily hours were summed for -each 
activity, then added across activi- 
ties for each student. This was the 
pr imary means of r epor t ing ac t iv i- 
ties offered, numbers served, and 
total Otro Camino student service 
hours. 

These data are tabulated in 
Tables 1 and 2. The tables show 
that clubs are the most intensive 
service provided by Otro Camino. 
Students who participated in clubs 
spent 25-30 hours per semester, or 
close to two hours per week, in 
these activities* The intensity of 
the vocational and academic services 
increased from semester one to sem- 
ester two, and the counseling ser- 
vices remained relatively low. The 
main counseling service was group 
counseling for 81 students who appa- 
rently met approximately once a 
month. 

Tables 1 and 2 show that, alt- 
hough the absolute number of stu- 
dents served declined from the first 
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to the second semester, the inten- 
sity of service increased. During 
the first semester 74.8% of the stu- 
dents participated in only one act- 
ivity — clubs, most likely. But dur- 
ing the second semester more than 
two-thirds of the students were 
active in two or more different 
activities. The average hours of 
participation per student also 
increased somewhat during the second 
semester. 

Outcomes 

Evaluation Design 

Otro Camino has an open door 
admission policy, meaning anyone who 
comes for services is served. Ran- 
dom selection of program partici- 
pants would have drastically altered 
this policy. A nonequivalent com- 
parison group design was estab- 
lished. This design enabled the 
comparison of participating and 
non-participating students from the 
targeted grades at each of the three 
program schools. Measures from 
Spring, 1981 SAES surveys and school 
record data were used to statisti- 
cally control for the nonequi> alence 
of the groups. 

Data 



The project was specifically 
designed to help students in the 
major transition grades 7, 9, and 
12, so data were collected on all 
students in the 1981 pre-target 
grade. Pre-target year and target 
year data include the SAES student 
questionnaire (administered in the 
spring of 1981 and spring of 1982), 
school records data on school atten- 
dance and grades, Puerto Rico Stand- 
ardized Test Scores, and records of 
police arrests for both years. 
These data were also reported for 
samples of all other grades between 
6 and 12, 



The SAES measures are described 
in Gottf redson, Ogawa, Rickert, & 
Gottf red^^i (1982). Two distinct 
scoring procedures were used. The 
1981 questionnaire scales mostly 
involved sums of standard scores for 
items in a scale. For the 1982 ver- 
sion, a new scoring procedure was 
implemented to make interpretation 
easier: Most scale scores reflect 
the proportion of items answered in 
the desirable direction. Because of 
this scoring difference and because 
the 1981 scale scores were used only 
as statistical control s, the 1981 
scores are not reported here. 

Analysis Procedures 

Measures of intensity of treat- 
ment were computed as follows: 
Records of treatment hours were col- 
lapsed into major categories: Aca- 
demic assistance activities, voca- 
tional development activities, club 
activities, counseling activities, 
and participation in activities dur- 
ing 1980-81 (the first year of pro- 
gram operation). 

The number of participation hours 
in each category was summed for both 
semesters and correlated with each 
outcome measure. The correlation 
matrix was examined for correlati- 
ions significant at the p<.U5 level 
(two-tailed) . 

Then, regression equations were 
used to estimate project effect 
parameters — that is the part of the 
correlation that can be attributed 
to participation in the program 
rather than selection effects. 
These equations included as statis- 
tical controls any pre-target year 
measures that correlated with both 
the activity category and the out- 
come . ^ Since patterns of correla- 
tions were different for various 
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activities, different control varia- 
bles were used in various tests. 
Notes on tables of results indicate 
the control variables for activity 
categories. 

Results 

Tables 3 through 6 present the 
correlation and regression coeffi- 
cients for all outcomes. Outcomes 
are organized according to project 
goals (Tables 3 . and 4), project 
objectives (Table 5), and other 
theoretically relevant objectives 
(Table 6). Appendix tables show 
means and standard deviations for 
all outcomes by activity category 
for persons participating in each 
kind of activity. 

Academic Activities 

Twenty-eight of thirty-nine cor- 
relations (72%) indicated the asso- 
ciation between participation in 
academic activities and academic 
outcomes was in the unfavorable 
direction. (Read across the rows of 
Tables 3-6.) Participant achieve- 
ment was lower than nonparticipant 
achievement on 12 of 13 measures 
(Table 3); participants reported 
skipping more classes (Table 4), 



^ In technical terms, a model of the 
form Y - ^^"i'^ t ^ estimated 
for each outcome variable (y)> 
where J is the standardized regres- 
sion coefficent for hours of parti- 
cipation in activity category (x), 
the are the standardized regres- 
sion coefficients for the exogenous 
control variables _f , and e is ran- 
dom error. The specification of 
each such model (i.e, the choice of 
the ^ ) was made to include as 
exogenous control variables any 1981 
measure correlated both with extent 
of participation in activity x and 
with the particular y under consid- 
eration. 



more al ienation, lower self-con- 
cepts , less interpersonal compe- 
tency , and spending less effort on 
school work than nonpar ticipants 
(Table 5); and participants reported 
more del inquent activity (self-re- 
port and record of arrests^, more 
victimization, more suspensions , 
less attachment to parents, more 
nega t ive peer inf luence , lower 
belief in the validity of conven- 
tional rules , less practical know- 
ledge , less internal control , and 
more school punishment and fewer 
school rewards than nonpar ticipants 
(Table 6). Most of the correlations 
were small and nonsignificant, but 
eleven were statistically signif i- 
cant. 

These uncontrolled associations 
are not unexpected: Persons seeking 
academic assistance in the program 
may reasonably be expected to be in 
need of it. These results do not 
imply that the participation/in. Otro 
Camino's academic activities brought 
about these outcomes. 

To examine whether these associa- 
tions remain significant when pre- 
existing differences among students 
are statistically controlled, the 
betas in Tables 3 through 6 must be 
inspected. When controls are 
applied, number of hours of partici- 
pation in academic activities is 
associated only with arrest and 
school punishment. In the context 
of the model for the data examined 
here, this association could be 
interpreted as an effect of partici- 
pation in Otro Camino's academic 
activities. Two of these betas are 
significant. 

Vocational Activities 

Thirty-seven of forty-one corre- 
lations (90%) between participation 
in vocational activities and out- 
comes were in the* favorable direc- 
tion. Only on reports of having a 
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job (Table 3), amount of school 
involvement, internal control, and 
school rewards (Table 6) were the 
nonpar ticipants favored- Twenty-one 
of these uncontrolled correlations 
are significant , but when statisti- 
cal controls are applied, only two 
remain significant. In the context 
of the models of the data examined 
here the results (shown in Table 3) 
could be interpreted as implying 
that participation in Otro Camino's 
vocational activities ' increased 
scores on a standardized test of 
Spanish language and increased 
grades (according to self-repor t ) 

Club Activities 

Thirty-four of forty-one correla- 
tions (83%) between participation in 
clubs and outcomes were in the favo- 
rable direction. Only for self-re- 
port of reading ability (Table 3), 
repor ts of victimization, suspen*- 
sions, belief in the validity of 
conventional rules, rebellious 
autonomy, and internal control 
(Table 6) were the nonparticipants 
favored. Fifteen of these correla- 
tions were significant. When sta- 
tistical controls were applied, 
three remained significant. In the 
context of the models examined, par- 
ticipation in clubs increased social 
science grades (Table 3), students' 
reports about student influence in 
the school^ (Table 5), and reports 
of involvement in school activities 
(Table 6), 



Although not significant when sta- 
tistical controls are applied, the 
pattern of results for actual school 
grades accords with self-report 
results. 

This measure is composed of items 
ordinarily scored at the school 
level (Gottf redson, Ogawa, Rickert, 
& Gottfredson, 1982) but combined 
here into an individual-level 
"scale" the psychometric properties 



Counseling Activities 

Thirty of forty-one correlations 
(73%) between counseling activities 
and outcomes were in the favorable 
direction, Nonparticipants v;ere 
favored on math grades, school grad- 
uation, having a job (Table 3), 
self -reports of school non-atten- 
dance (Table 4) , al ienation (Table 
5), reports of victimization, sus- 
pensions, belief in the validity of 
conventional rules, and practical 
knowledge (Tabl e 6 ) , ,The appl ica- 
tion of statistical controls reduced 
the number of significant correla- 
tions from six to two. In the con- 
text of the models examined here, 
'> the results imply that participation 
in Otro Camino counseling activities 
increases educational expectations 
(Table 3) and involvement in school 
activities (Table 6), 

A re-examination of academic 
activities . Project managers were 
asked about the academic activities 
results. They hypothesized that the 
students referred for ■ intensive 
tutoring may have been markedly 
different from students voluntarily 
participating in Otro Camino's other 
academic activities. Furthermore 
the tutoring came late in the year 
and the nature of this intervention 
was different from, other academic 
activities. They requested that 
academic activities be re-examined 
excluding the tutoring activity, so 
the results could be used in helping 
decide which way to approach aca- 
demic activities in the comivjg year. 
When tutoring is excluded and only 
those academic activities indicated 
by the absence of an <a> in Table 1 
are examined, there is no dependable 
indication of any effect (positive 
or negative) of participation in the 
remaining academic components. 



of which have not been examined. 
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Discussion 

Except for results associated 
with participation in academic 
activities, the results are encour- 
aging. Most of the correlations are 
in the right direction and there is 
some evidence that participation had 
positive effects. However, the 
actual difference between partici- 
pants and nonpar ticipants is small. 
For instance , less than 1% of the 
variance in Spanish language stand- 
ardized test scores is accounted for 
by participation in vocational 
activities. Most of the variance is 
accounted for by grades and ability 
measured by standardized test scores 
in the previous year. 

The findings cannot necessarily 
be attributed to participation in 
the specific activity category 
listed because many students (about 
78%) participated in more than one 
activity. The analytic procedures 
used included only hours of partici- 
pation in one category at a titn'^ as 
a means of helping project managers 
learn what happened for the various 
categories of activities imple- 
mented. 

Project participation was not 
under experimental control; thus it 
is possible that the results reflect 
self-selection or differential matu- 
ration. Participants were plausibly 
more highly motivated to better 
themselves and measures applied as 
statistical controls may have been 
insufficient to control for this 
possibility. 

A final word of caution in the 
interpretation of the significant 
results from the regression equa- 
tions: Some significant differences 
can be expected to ar ise by chance . 
In these analyses, about two of the 
39 or 41 equations examined for each 
category of participation would be 
expected to reach significance at 
the p<.05 level simply by chance . 



This is precisely the number of sig- 
nificant differences observed. 

In support of the argument that 
the results indicate, the effects of 
participation, many of the resul ts 
ate sensible relative to project 
activities. Project intervention 
with students referred for intensive 
tutoring did not begin until near 
the end of the school year, allowing 
plenty of time for academic and 
behavior problems to occur before 
participation and af ter pre-level 
measures used as statistical con- 
trols were collected. The reasons 
for their school referral included 
recent discipl inary problems, 
"declining achievement, and- atten- 
dance problems. 

Several of the vocational devel- 
opment activities emphasized lan- 
guage skills and exam preparation. 
It is not surprising that standard- 
ized test scores --improved -for -this 
group. Students in this group were 
al so reported to have renewed their 
commitment to education in prepara- 
tion for careers. 

Many clubs emphasized the commu- 
nity culture and history. Others 
were aimed specif ical ly at devel op- 
ing leadership skills. It is sensi- 
ble that students in these activi- 
ties had higher social science 
grades and perceived more student 
influence at the school than the 
nonparticipants. On the other hand, 
people who are "joiners" in one set- 
' ting are likely to be "joiners" in 
an o th er se 1 1 ing . Thus it is no t 
surprising to find Otro Camino par- 
ticipants also participated in a 
variety of school activities, and 
this result may therefore reflect 
primarily a selection artifact. 

The counseling component included 
helping students decide about future 
plans and determine the education 
needed to successfully reach their 
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personal goals. Counselors 
emphasized the need to stay in 
school — both staying in school now 
and pursuing education in the 
future. Much of the counseling 
occurred at the high school where 
many of the students who were going 
to leave school had already with- 
drawn and at a time in students^ 
1 ives when decisions about the 
future must be made. It is sensible 
that this component is associated 
with higher educational expecta- 
tions. 

Finally, I have observed that the 
entire project staff are committed 



to the goals, theory, and objec- 
tives, and implement planned compo- 
nents with integrity. They have 
encountered numerous obstacles, and 
implementation of some components 
has been seriously delayed. But 
each delay and each obstacle has 
resulted in more planning, and these 
efforts appear systematically to 
strengthen the component while 
remaining true to the theory that 
led to their development. Perhaps 
the most serious implementation 
problem the project woqld ever face 
is delay while new strategies are 
developed to insure implementation 
of components anchored in theory. 
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Table 1 



Number of Students Served and Intensity 
of Treatment — by Activity and Semester 
During the 1981-82 School Year 







Semester 1 






Semester 2 






Number 


of Avg. hrs./ 


% Total 


Number of 


Avg. hrs./ 


% Total 


itivity'^ 


-students student 


effort 


students 


student 


effort 


lademic 


157 


11.2 


13.3 


115 


22.7 


27 .0 


Oral Commun i c a t i on — 


37 


5 .5 


1 .5 


8 


6.4 


.5 


English 










7.1 


.5 


Reading — Engl ish 


16 


5.9 


.7 


7 


Speak, Read and 


13 


. 5.8 


.6 


0 


0.0 


0.0 


V7rite — English 












0.0 


Keeping with the 


1 


2.0 


0.0 


1 


1 .0 


News — Spanish 










8.5 


.2 


Learning to Read 


1 


24.0 


.2 


2 


and Write — Spanish 










0.0 


0.0 


Growing with Music 


16 


8.8 


1.1 


0 


— Spanish 










0.0 


0.0 


The Why of Math — Math 


6 


3.7 


.2 


0 


Multiplying to 


2 


2.0 


0.0_ 


_ o_ _ 


_o.o_ 


... o.p 


Divide — Math 










2.8 


.2 


Everyday Problems 


21 


3.4 


.5 


6 


—Math 














Photography — Science 


30 


28.9 


6.6 


20 


29.8 


6.3 


Puerto Rico in the 


2 


30.0 


.5 


0 


0.0 


0.0 


World — Social Science 










0.0 


Learning More 


14 


13.0 


1.4 


0 


0.0 



About Puerto Rico — 
Social Science 
Civics^ 

Natural Science^ 
English^ 
Spanish^ 
Math^ 

Social Science^ 
Investigation Club 



0 


0.0 


0.0 


26 


8.5 


2.3 


2 


2.0 


0.0 


37 


8.5 


2.3 


0 


0.0 


0.0 


38 


5.7 


2.3 


0 


0.0 


0.0 


39 


7.0 


2.9 


0 


0.0 


0.0 


43 


6.2 


2.8 


0 


0.0 


0.0 


38 


7.3 


2.9 


0 


0.0 


0.0 


5 


71.0 


3.8 



Note. "Number of students" column reports the number of students who had at least 
one hour of participation in a given activity. The "average hours per student" 
column reports the average number of hours for students who participated for at least 
one hour in the activity. The "% total effort" column reports the total number of 
hours spent by all students as a ratio of the total number of participation hours. 
^ Second semester participation hours in these activities represent time in Otro Cam- 
ino intensive tutoring; all other courses in the academic group represent creative 
academic activities. 

All 1981-82 activities except the Summer, 1982 "Mini-University" are included on 
this table. 

(continued on next page) 
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Table 1 (cont.) 







Semester 1 






Semester 2 






Number of 


Avg. hrs./ 


% Total 


Number of 


Avg. hrs./ 


/o Total 


« . • • . b 
Activity 


students 


student 


effort 


students 


student 


ef fort 


Vocational 


216 


17 .9 


29.4 


43 


A 


11 Q 
11.7 


Librarian Aides 


1 n 


AQ ft 




7 


36 4 


2.7 


Audiovisual 


75 


4.0 


2.3 


4 


1 Q A 
1 J .0 


.b 


Techniques 










V .u 


u . u 


My Career for the 


4 


1 / A 

14.0 


.4 


A 
U 


Future 














Getting Prepared 


20 


29.1 


4.4 






ft A 
















Where Will I Study 


8 


23.3 


1.4 


0 


0.0 


0.0 


College Entrance 


103 


21.1 


16.5 


0 


0.0 


0.0 


Exam Preparation 










0.0 


A f\ 

0.0 


Short Term Careers 


8 


10.1 


.6 


0 


Club 














Clubs 


269 


24.7 


51 .3 


156 


30.0 


/AT 

49.7 


Public Relations 


10 


2.0 


• 2 


1 o 


.z 


J .u 


Education 


0 


0.0 


0.0 


0 


0.0 


A A 


Sports 


0 


0.0 


0 .0 


0 


A A 


u .u 


Model Building 


13 


4.9 


.5 


o 

J 


O A 

z .0 


1 

.1 


Dramatic Arts Club 


10 


94.0 


T 1 

7 .1 


1 J 


7 ^ 
/ .J 


1 n 
1 .u 


Aesthetic and 


. 48 


16.8 


7.1 


20 


26 .5 


5.0 


Modeling Club 








1 A 
14 


zu • y 




Astronomy Club 


23 


55.2 


A T 

9 .7 


J • 1 


. Gardening Club 


17 


14.9 


1 .9 


12 


11.9 


1 c 

1 •D 


Handicrafts for 


57 


20.6 


8.9 


o o 


lb .J 


A ft 


Girls 














Handicrafts Club 


46 


40.8 


14.3 


17 


CO o 

53 .2 


A £ 

9 .b 


Folic DflncpQ of 


27 


4.5 


.9 


8 


60.1 


5.1 


La PI ay a 








Q 
O 


9ft n 
zo . u 




Folklore of La Playa 


57 


O 1 

2.1 


A 




Presidents Club 


U 


u .u 


u . u 




0 n 


0.0 


Karate Club 


0 


0.0 


0.0 


15 


lb .4 


9 ^ 

z .b 


Vocal Group 


0 


A A 

0.0 


A A 


1 n 

lU 


ft9 Q 
OZ • y 


ft ft 


r!oiin qpI 1 


117 


7.6 


6.8 


140 


7.2 


10.7 


Self Development 


14 


16.0 


1.7 


0 


0.0 


0.0 


Club 














Individual 


22 


12.5 


2.1 


33 


2.0 


.7 


Counseling 








111 


8.5 


10.0 


Group Counseling 


81 


4.8 


3.0 


Total 


653 


20.1 


100.0 


397 


23.7 


100.0 
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Table 2 



Number of Students Participating in 
Otro Camino Activities — by Semester 
of the 1981-82 School Year 



Number of 

Concurrent Semester 1 Semester 2 

Activities N % N % 



1 


493 


74.8 


136 


31.9 


2 


142 


21.5 


122 


• 28.6 


3 


. 21 


3.2 


56 


13.1 


More than 3 


3 


0.5 


112 


26.3 


Total<a> 


659 


100.0 


426 


99.9 


Note. Numbers 
1980-81 school 


do not include 
year activities 


students who 


participated 


only in 



<a> The unduplicated number of students served ^f or the 1981j282 
school year is 797. 
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Tabic 3 



Correlation Coefficients and Standardized Regression Coefficients for 1981-82 
Academic Achievement Outcomes with Number of Hours in Each Type of Otro Camino Activity 



School Grades 



Project Goal I 
Standardized Tests 



Self Report 



Self Reporl 



Activity 



Span 



Eng 



Math 



^oc. 
Sci. 



Sci. 



Span. 
Read. 



Span . 
Lang. 



Eng. 
Read. 



Math 



Sch. 
Grade 



Read. 
Abil. 



Educ. 
Expect . 



Have 
Grad.<b> Job 



Paid 
Work 



Academic 
r 

beta 


-.041 
— <c> 


-.027 
— <c> 


-.053 
— <c> 


-.005 
— <c> 


.017 
— <c> 


-.027 
— <c> 


.011 
-<c> 


-.063* 
-.020 


-.024 
— <c> 


.060* 
.020 


.016 

— <c> 


-.048 
— <c> 


-.013 
— <c> 


.005 
— <c> 


.020 
— <c> 


Vocat ional 
r 

beta 


.164* 
.055 


.145* 
.023 


.043 
— <c> 


.039 
— <c> 


.100* 
.009 


.095* 
.032 


.200* 
.083* 


.071* 
-.025 


.201* 
.077 


.180* 
.084* 


.044 
— <c> 


.163* 
.096 


.442* 
.050 


-.045 
— <c> 


.039 
— <c> 


Clubs 
r 

beta 


.059* 
-.014 


.072* 
.003 


.073* 
.017 


.140* 
.069** 


.094* 
.023 


.029 
— <c> 


.056* 
.014 


.042 
— <c> 


.030 
— <c> 


.087* 
.025 


-.014 
— <c> 


.009 
— <c> 


-.032 
— <c> 


.020 
— <c> 


-.009 
— <c> 


Counseling 
r 

beta 


.046 
— <c> 


.056* 
-.011 


-.018 
— <c> 


.088* 
.031 


.049 
— <c> 


.008 
— <c> 


.054 
.-<c> 


.054 
— <c> 


.054 
— <c> 


.072* 
.013 


.022 
— <c> 


.118* 
.090** 


--.038 
— <c> 


-.034 
— <c> 


-.033 
— <c> 


Participation 




























• 




1981 
r 

beta 


.010 
— <c> 


.019 
— <c> 


-.014 

— <c> 


-.101* 
-.034 


-.031 
— <c> 


.092*<d> 
.145** ■ 


.038 
-<c> 


.045 
— <c> 


.091*<d> 
.100* 


.036 
— <c> 


.006 
— <c> 


.043 
— <c> 


.064* 
-.017 


.028 
— <c> 


.036 
— <c> 



I 

CO 
CO 
00 

I 



Note. Participation 1981 is a dummy variable measuring whether or not students participated m i^oi-^^ ratner tnan uuu^uy 
'^s N's range from 447 to 1006. depending on the patterns of missing data for variables in each equation. Analyses for eac 
outcome variable uses 1981 variables that are significantly correlated with that outcome and with participation as statistical 
control variables. Academic activity controls include 1981 standard test English reading score; English grade; and SAES friend 
influence scale. Vocational activity controls include 1981 standard test' math and Spanish reading scores; school days absent, 
scbo^grades f^^ Spanish! English, social science, and science; overall , grade point average; and SAES belief, interpersonal co 
petency! sc^^^^^ index! school punishment index, self-concept, rebellious autonomy, school attachment, total delinquency 

Hrlous delinquency, and victimzation scales; self-report of gender; and 1981-82 school grade. Club activity controls nclude 
1981 standard test Spanish language score; school days absent, and SAES self-report of grades, attachment to parents, interper- 
sonal competency, school punishment index, attachment to school, victimization, and self-report of ^^^^^-"^"^"'^^^ 
Counseling activity controls include 1981 standard test English reading score; school grade for math ^"^S^^ 
internal control and rebellious autonomy scales. Participation 1981 controls include 1981 school grade for English, and SAES 

rebellious autonomony, attachment to school, and self-report of socio-economic . level scales. 



interpersonal competency. 

<a> See Table 1 for listing of specific Otro Camino activies for each category. 
<b> Reported for 1981-82 twelfth-grade students only. 

<c> Regression coefficient not reported because activity type is not. related to outcome. 
<d> Beta unstable; interpret with catition. , 

* Coefficient is significant at the P<.05 level. ' 
**Coef f icient is significant at the P<.01 level. 
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Table A 



Correlation Coefficients and Standardized Regression Coefficients 
for 1981-32 School Withdrawal and Attendance Outcomes 
with number of Hours in Each Type of Otro Camino Activity 







Pr 0 j e c t 


Prtal IT nnrf III 








School 


Report 


Self 


Report 








Percent 


School 


Skipping 


Skipping 


Activ ity Type<a> 


Withdrawal<b> 


Days Absent<c> 


Attendance<d> 


School 


Classes 


Academic 










.022 


r 


-.048 


— <f> 


-.010 


-.021 


beta 


— <e> 


— <f> 


— <e> 


— <e> 


— <e> 


Vocational 












r 


-.057* 


-.094*, 


-.000 


-.033 


-.022 


beta 


— <f> 


.024 


— <e> 


— <e> 


— <e> 


Clubs 










-.076* 


r 


-.063* 


-.090* 


-.072* 


-.052 


beta 


-.036 


-.036 


-.048 


— <e> 


-.046 


Counse 1 ing 










-.035 


r 


-.049 


-.040 


.002 


.010 


beta 


— <e> 


— <e> 


— <e> 


— <e> 


— .<e> 


Participation 1981 










-.054 


r 


.045 


.021 


-.035 


-.031 


beta 


— <e> 


— <e> 


— <e> 


— <e> 


— <e> 



Nfi.te. Participation 1981 is aduinmy variable measuring whether or not students par- 
ticipated in 1981-82 rather than number of hours. N's range from 447 to 1006, 
depending on the patterns of missing data for variables in each equation. Each out- 
come is controlled by 1981 variables that are significantly related to the type of 
activity. Academic activity controls include 1981 standard test English reading 
score; English grade; and SAES friend's influence scale. Vocational activity con- 
trols include 1981 standard test math and Spanish reading scores; school days 
absent; school grades for Spanish, English, social science, and science; overall 
grade point average; and SAES belief, interpersonal competency, school rewards 
index, school punishment index, self-concept, rebellious autonomy, school attach- 
ment, total delinquency, serious delinquency, and victimization scales; self-report 
of gender; and 1981-82 school grade. Club activity controls include 1981 standard 
test Spanish language score; school days absent, and SAES self-report of grades, 
attach:nent to parents, interpersonal competency, school punishment index, attachment 
to school, victimization, and self"-report of socio-economic level scales. Counsel- 
ing activity controls include 1981 standard test English reading score; school grade 
for math and social science; and SAES internal control and rebellious autonomy sca- 
les. Participation 1981 controls include 1981 school grade for English, and SAES 
interpersonal competency, rebellious autonoraony, attachment to school, and self-re- 
port of socio-economic level scales. 

<a> See Table 1 for listing of specific Otro Camino activies for each category. 

<b> Reasons for withdrawal include voluntary and involuntary withdrawal, transfer to 

another school, and reason unknown. 

<c> Percent based on number of days active. 

<d> Overall school and class attendance. 

<e> Regression coefficient not reported because activity type is not related to out- 
come . 

<f> Not analyzed. 

* Coefficient is significant at the P<.05 level. 
**Coef f icient is significant at the P<.01 level. 
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Table 5 . 

Correlation Coefficients and Standardardized Regression Coefficients 
for 1981-82 SAES Student Questionnaire Scales with Number of Hours 
in Each Type of Otro Camino Activity 



Project Objectives 



School 

Activity type<a> Alienation Attachment 



Self Interpersonal School Student 
Concept Competency Effort Influence 



Acadenic 



beta 



.031 
-<b> 



.007 
-<b> 



.072* 
.044 



.035 
-<b> 



.066* 
.039 



— <c> 
— <c> 



Vocational 
r 

beta 



.053 
-<b> 



.054 
-<b> 



.106* 
.060 



.059* 
.038 



.124* 
.067 



.092* 
.028 



Clubs 
r 

beta 



.031 

-<b> 



.076* 
.021 



.059* 
.013 



.026 

— <b> 



.077* 
.038 



.100* 
.lOi** 



Counseling 
r 

beta . 



.009 
-<b> 



.021 

-<b> 



.055 

-<b> 



.036 
-<b> - 



.072* 
- .044 - 



.044* 

— <b>- 



Participation 81 

r -.005- 
beta — <b> 



.027 

-<b> 



.051 

-<b> 



.016 

-<b> 



.012 

— <b> 



.;06* 

.086** 



l^ote. Participation 1981 is a dummy variable measuring- whether or not students parti- 
cipated in 1981-82 rather than number of hours. N's range from 447 to 1006, depending 
on the patterns of missing data for variables in each equation. Each outcome is cont- 
rolled by 1981 variables that are significantly related to the type of activity. Aca- 
demic activity controls include 1981 standard test English reading score; English 
grade; and SAES friend's influence scale. Vocational activity controls include 1981 
standard test math and Spanish reading scores; school days absent; school grades for 
Spanish, English, social science, and science; overall grade point average; and SAES 
belief, interpersonal competency, school rewards index, school punishment index, self- 
concept, rebellious autonomy^ school attachment, total delinquency, serious delin- 
quency, and victimization scales; self-report of gender; and 1981-82 school grade. 
Club activity controls include 1981 standard test Spanish language score; school days 
absent, and SAES self-report of grades , attachment to parents, interpersonal compe- 
tency, school punishment index, attachment to school, victimization, and self-report of 
socio-economic level scales. Counseling activity controls include 1981 standard test 
English reading score; school grade for math and social science; and SAES internal con- 
trol and rebellious autonomy scales. Participation 1981 controls include 1981 school 
grade for English, and SAES interpersonal competency, rebellious autonomony, attachment 
to school , and sel f-repor t of socio-economic level scales . 

<a> See Table 1 for listing of vhich Otro Camino activities are included in each cate- 
gory. 

<b> Regression coefficient is not reported because activity type is not related to out- 
come. 
<c> Not examined. 

* Coefficient is significant at the p<.05 level. 
**Cocf f icient is significant at the p<.01 level. 
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Tflblc 6 (continued) 



Cotrflatiun Cot>f f icii^nts and Stondurdiscd Rcgrcsaion Coefficients for 
1981-82 Studi-nt QucBtionnn Ire Scales with Number of Hours in Each type 
of Otro Camino Activity 



Other Theoretically Relevant Outcomes 



Attil\)trn and 5>ocinl Dove) opmcnL<b> School Experiences 

Belief in Practical Rebellious Internal 

Activity typc<A> Rules Involvement Knowledgu Autonomy Control Punishment Rewards 



Academic 

r -.062* 

tela -.047 

Vpcational 

r .096* 

bQta -.031 

Chibi 

r -.012 

beta — <c> 

Counse 1 Ing 

r -.006 

beta — <c> 

Participation 81 

r .039 

beta — <c> 



.027* -.055* V -.045 

— <c> -.043 — <c> 

-.013 .086* -.044 

— <c> .063 — <c> 

.067* .007 .007 

.085* — <c> — <c> 

.081* -.024 -.009 

.083* — <c> ~<c> 

.083* .033 -.023 

.085 — <c> — <c> 



-.014* .111* -.006 

— <c> .099** — <c> 

-.008 -.107* -.009 

— <c> -.058 — <c> 

-.009 -.024 .051 

— <c> — <c> — <c> 

.025 -.021 .029 

-.-<c> — <c> — <c> 

-.041 .043 .052 

— <c> — <c> — <c> 



Note. Participation 1981 is a dummy variable measuring whether or not students participated in 
198l-b2 rather than number of hours. N's range from 447 to 1006, depending on the patterns of missing 
data for variables in each equation. Each outcome is controlled by 1981 variables that are signifi- 
cantly related to the type of activity. Academic activity controls include 1981 standard test English 
reading score; English grade; and SAES friend's influence scale. Vocational activity controls include 
1981 standard test math and Spanish reading scores; school days absent; school grades for Spanish, 
English, social science, and science; overall grade point average; and SAES belief, interpersonal com- 
petency, school rewards index, school punishment index, self-concept, rebellious autonomy, school 
attachment, total delinquency, serious delinquency, and victimization scales; self-report of gender; 
and 1981-82 school grade. Club activity controls include 1981 standard test Spanish language score; 
school days absent, and SAES self-report of grades, attachment to parents, interpersonal competency, 
school punishment index, attachment to school, victimization, and self-report of socio-economic level 
scales. Counseling activity controls include 1981 standard test English reading score; school grade 
for math and social science; and SAES internal control and rebellious autonomy scales. Participation 
1981 controls include 1981 school grade for English, and SAES interpersonal competency, rebellious 
autonowony, attachment to school, and self-report of socio-economic level scales. 

<a> See Table 1 for listing of which Otro Camino activities are included in each category. 

<b> See SAES Student Questionnaire Manual for explanation of these variables. 

<c> Regression coefficient is not reported because activity type is not related to outcome. 

* Coefficient is significant at the p<.05 level. 
**Coef f icicnt is significant at the p<.01 level. 
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Table A 



Means and Standard Deviations for 1981-82 Academic Achieveraent 
Outcomes for Each Type of Otro Camino Activity 



Activity 
Type^ 



School Grades 



Outcomes for Project Goal 1 
Standardized Tests 



Self-Reported 



Soc . 

Span. Eng. Math Sci. Sci. 



Vocation 



Span. 
Read. 


Span. 
Lang. 


Engl. 
Read. 


Math 


Schl. 
Grade 


• Read. 
Abil. 


H.S. 

Grad.D 


Have 
Job 


Pa 
Wo 


39.74 


34.17 


30.16 


20.32 


2.11 


1.52 


.12 


.48 


.2 


12.86 


10.80 


12.38 


7.62 


.93 


.87 


.33 


.73 


.4 


197 


195 


196 


203 


201 


200 


236 


199 


19 


41.32 


37.12 


30.11 


23.38 


2.43 


1.56 


.52 


.20 


.2 


13.48 


12.71 


10.41 


8.21 


.98 


.83 


.50 


.52 


.4 


203 


202 


203 


209 


194 


•193 


231 


192 


19 


39.73 


34.07 


29.59 


18.76 


2.25 


1.47 


.08 


.32 


.2 


13.25 


11.37 


11.58 


6.87 


.85 


.89 


.28 


.62 


.4 


313 


302 


304 


308. 


30^ 


300 


362 


302 


3C 


38.34 . 


33.59 


29.82 


18.32 


2.14 


1.47 


.03 


.32 


.2 


13.74 


11.30 


12.23 


6.78 


.86 


.9.2 


.17 


.62 




208 


209 . 


1208 


214 


210 


207 


246 


208 


2( 


42.03 


34.96 


1 

'29:67 


21.50 


2.22 


1.54 


.21 


.35 




14.23 


11.91 


11.82 


8.04 


.86 


.91 


.41 


.66 


.i 


117 


116 


* ill6 


120 


110 


112 


157 


111 


X 



Academic 
Mean 
SD 



Vocational 
Mean 
SD 
N 

Clubs 
Mean 
SD 
N 

Counseling 
Mean 
SD 
N 



Participation 
1981 
Mean 
SD 
N 



1.66 
1.01 
231 



2.10 
1.06 
229 



1.70 
.95 

354 



1.64 
.95 

244 



1.80 
1.08 
149 



1.69 
1.05 
230 



2.10 
1.10 
226 



1.86 
1.00 
355 



1.73 
1.01 
240 



1.84 
1.05 
148 



2.04 
1.06 
213 



2.24 
1.07 
182 



2.17 
.96 

337 



2.06 
.97 

234 



2.16 
.98 
131 



1.73 
1.06 
202 



1.80 
1.06 
124 



1.86 
.92 

326 



1.71 
.95 

234 



1.41 
1.03 
110 



1.74 
1.02 
195 



1.89 
1.03 
129 



1.83 
.93 

309 



1.75 
1.00 
221 



1.62 
1.00 
115 



^ See Table 1 for listing of specific Otro Camino activities for each category. 

0 j 

^ Reported for 1981-82 twelfth-grade students only. 
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Table B 



Means and Standard Deviations for 1981-82 
^ School Withdrawal and Attendance 
for Each Type of Otro Camino Activity 





School 


Report 




Self Report 




Activity type^ 


With- 
drawal^ 


Percent 

days 
absent*^ 


School 
nonatten- 
dance 


Skipping 
school 


Skipping 
classes^ 


Academic 
Mean 
SD 
N 


.042 
.202 
236 


.024 
.056 

235 


1.116 
.740 
199 


.658 
1.084 

199 


1.213 
1.012 
202 


Vocational 
^lean 
SD 
N 


.056 
.231 
231 


.033 
.135 
230 


'.J 

1.163 
.767 
196 


.801 
1.140 

196 


1.162 
.960 
197 


Clubs 
Mean 
SD 
N 


.044 
.206 

362 


.025 
.080 

362 


1.107 
.704 

307 


.610 
1.017 
308 


1.107 
.883 
308 


Counsel ing 
Mean 
SD 
N 


.037 
.188 
246 


.023 
.077 
244 


1.165 
.745 
212 


.722 
1.090 
212 


1.122 
.903 
213 


Participation 81 
Mean 
SD 
N 


.108 
.312 

157 


.055 
.159 

157 


1.133 
.773 
113 


.743 
1.132 
113 


1.044 
.876 
114 


^ See Table 1 for 


listing of 


which Otro 


Camino activities are included in 



each category. 

Reasons for withdrawal include voluntary and involuntary withdrawal, trans- 
fer to another school, and reason unknown. 
^ Percent based on number of days active. 

Individual item ranges from 0 to 4 with 0 = no days cut in the last four 
weeks and 4 = more than 10 days cut in the last 4 weeks. 
^ Individual item ranges from 0 to 5 , with 0 = never cuts classes and 5 = 
cuts classes almost every day. 
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Table C 



Means and Standard Deviations for 1981-82 SAES Student Questionnaire 
Scales for Each Type of Otro Camino Activity 



Project Objectives 



Alienation 



School 
Attachment 



Self 
Concept 



Interpersonal 
Competency 



School 
Effort 



Academic 
Mean 
SD 



.346 
.227 

177 



.762 
.212 
186 



.645 
.173 
161 



.637 
.268 
175 



.583 
.263 
181 



Vocational 
Mean 
SD 
N 



.299 
.222 
179 



.809 
.198 
189 



.700 
.180 

159 



.735 
.234 
180 



,623 
,280 
188 



Clubs 
Mean 
SD 
N 



.314 
.224 
272 



.775 
.209 
288 



.671 
.179 

237 



.683 
,278 
276 



.615 
.267 
289 



Counsel ing 
Mean 
SD 
N 



.316 
.215 
185 



.770 
.214 

199 



.684 
.173 
158 



.700 
.247 

183 



.584 
.279 
196 



Participation 81 

Mean .329 

SD .216 

N 93 



.762 
.223 
101 



.678 
.168 
84 



.700 
.242 
102 



.573 
.282 
106 



Note. See Table 1 for listing of which Otro Camino activities are included 
in each category. 
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Table D 



Means and Standard Deviations for 1981-82 Student Questionnaire 
Scales with Each Type of Activity 



Other Theoretically Relevant Outcomes 





Self Report of Delinquency^ 




Social Relations 


Attitudes and 


Soc. 


DvlpmnL . 


Schl. 


Expe 


Activity 
Type^ 


Total 
Del in. 


Ser. 
Del in. 


Drug 
Usage 


Victm. 


School 
Suspn. 


Pol ice 
Cntct. 


Attch. 

to 
Prnts. 


Neg. 
Peer 
Influ. 


Belief 

in 
Rules 


Invol . 


Pract . 
. Knowl. 


Rebel 
Auto- 
nomy 


. In- 
ternal 
Cntrl . 


Punish 


Rew 


Academic 
Mean 
SD 
N 


.069 
.117 
182 


.046 
.118 
181 


.119 
.192 
179 


.086 
.169 
182 


.158 
.366 
183 


.021 
.144 
236 


.642 
.262 
197 


.141 
.157 
177 


.778 
.209 
175 


.217 
.167 

179 


1.386 
.479 
179 


.345 
.322 
171 


.426 
.209 
176 


.101 
.193 
183 


.32 
.28 

185 


Vocational 
Mean 
SD 
N 


.056 
.116 
191 


.034 
.109 
191 


.106 
.195 
189 


.060 
.143 
188 


.086 
.281 
186 . 


.004 
.066 
231 


.661 
.256 
193 


.103 
.141 
187 


.841 
.178 
182 


.185 
.139 

183 


1.520 
.449 
183 


.369 
.319 
178 


.398 
.204 
176 


.048 
.145 
189 


.29 
.29 

190 


Clubs 
Mean 
SD 
N 


.053 
.110 
293 


.036 
.102 

293 


.082 
.173 

2&8 


.082 
.172 
287 


.156 
.363 
276 


.008 
.091 
362 


.669 
.266 
303 


.141 
' .172 
290 


.779 
.203 
277 


.230 
.181 

284 


1.4*36 
.456 
281 


.388 
.328 
273 


.423 
.219 

273 


.092 
.196 
286 


.31 
.32 

287 


Counsel ing 
Mean 
SD 
N 


.045 
.097 
192 


.031 
.090 
191 


.073 
.152 
191 


.072 
.156 
196 


.157 
.365 
197 


.008 
.090 
246 


.691 
.249 
209 


.145 
.171 
193 


.776 
.202 
185 


.226 
.192 

. 192 


1.465 
.450 
190 


.404 
.325 
182 


.429 
.225 
184 


.085 
.176 
196 


.29 
.29 

197 


Participation 
1981 
Mean 
SD 
N 


.065 
.123 
107 


.043 
.109 
107 


.116 
.205 
105 


.078 
.180 
100 


.173 
.381 
98 


.019 
.137 
157 


.651 
.273 
112 


.144 
.184 
107 


.782 
.231 
101 


.234 
.166 

105 


1.507 
.414 
101 


.367 
.360 
98 


.395 
.251 
99 


.124 
.233 
102 


.32 
.3C 
lOJ 



^ See Table 1 for listing of specific Otro Camino activities for each category. 
^ See SAES Student Questionnaire Manual for explanation of these variables. 
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Project PATHE: Second Interim Report 



D. C. Gottfredson 



Abstract 



Interim results suggest that PATHE, a school-based program that com- 
bines organizational change , direct service to high risk youths , and 
strong management, significantly increased academic performance for tar- 
geted individuals. In one of the four middle schools, target students 
reported significantly less delinquent behavior than did the control 
group* 

The program appears to have had a positive school-level effect on 
student alienation, self-concept, and belief in rules as well as on 
school safety, victimization, and teacher morale. Delinquent behavior 
was lov/er in all three PATHE high schools in 1982, although delinquent 
behavior was higher in the control high school. A nonsignificant trend 
in favor of the program is evident on several additional PATHE goals and 
obj ectives. 

Implementation data show much variation across program components and 
across schools in the fidelity of the implementation of the program. 
The expected association between level of implementation and success of 
the program was not, however, observed. Plans for strengthening the 
program and the monitoring procedures for the 1982-83 school year are 
discussed. 



PATHE — Positive Action Through 
Holistic Education — effectively com- 
bines school-wide organizational 
change, direct service to high-risk 
youths and rigorous management and 
evaluation. The program was devel- 
oped by the Charleston County School 
District, Charleston, S.C., and has 
been operating in seven middle and 
senior high schools for two years. 
This report updates the previous 
interim evaluation report for the 
PATHE program (D. Gottfredson, 
1982). The background information 
included in the first report, 
including descriptions of the 
Charleston community and the PATHE 
program, will not be repeated here. 
This report summarizes major program 
changes that occurred between Sep- 
tember, 1981 and August, 1982, 
reports results of analyses of 
implementation and outcome data for 
the same year , and discusses impli- 



cations of 
operations. 



the data for program 



Major Program Changes 
Staffing 

The PATHE staff remained rela- 
tively stable during the 1981-82 
school year . Both special ists at 
St. John's High school were replaced 
in August , 1981 , and the Student 
Concerns Specialist at Rhett Middle 
school was replaced mid-year.^ . The 
replacement specialists resembled 
their predecessors in qualifications 
and backgrounds and the staff 
changes seem not to have been dis- 
ruptive to the program. A cut-back 
in funding resulted in the lay-off 
of five special ists at the end of 
the 1981-82 school year, and the 
evaluator position also turned over 
at that point. The effect of the 
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reduction in staff at the schools 
for the 1982-83 year will be 
assessed in the final report for 
this project. The turnover in the 
evaluator position was smooth and 
caused no disruption to the program. 
The new evaluator is knowledgeable 
about research and evaluation tech- 
niques, and she is skilled in pro- 
gram management. 

Program Changes 

The goals, theory, and objectives 
of the program remained unchanged 
during the 1981-82 year. One new 
program component—Student Team 
Learning — was added and three — the 
Speakers Bureau, Summer Program to 
Access Creative Experiences, and the 
M.U.S.C. Career Awareness and Train- 
ing Programs — were dropped. 

In Spring, 1982, the program man- 
agers reorganized the 28 PATHE 
interventions into five major cate- 
gories of interventions. This 
reconceptual ization of the program, 
which amounted to combining inter- 
ventions that had previously been 
regarded as separate program compo- 
nents, helped to organize and focus 
program operations. A description 
of the PATHE interventions as reor- 
ganized follows. Descriptions of 
the components are provided only for 
those interventions which changed 
significantly or were added since 
the first interim report, or whose 
original descriptions required ela- 
boration. 

PATHE Organizational Structure 

This school level intervention is 
intended to establish and maintain 
an organizational structure which 
facilitates shared decision making 
among community agencies, students, 
teachers, school administrators and 
parents in the management planning 
for the school. The intervention 
provides training in assessing 



needs, researching problems, 
defining objectives, developing and 
implementing plans, assessing pro- 
gress and redef ining strategies. 
The sub-components of the interven- 
tion follow. 

Curriculum Review and Revision 
provides a structure for ongoing 
review and revision of the curricu- 
lum pol icies. It identifies train- 
ing and material needs, provides for 
faculty inservice training, and 
operates a "resource rocjm." 

Discipline Policy Review and 
Revision provides a structure for 
ongoing review and revision of the 
discipl ine pol icies. This includes 
the establishment and maintenance of 
a discipl ine referral system and 
strategies to generate and publicize 
school and classroom rules. This 
activity also identifies training 
and material needs, provides for 
faculty- inservice — training, - and 
identifies materials to be stocked 
in the "resource room." 

Leadership/Suppor t Teams receive 
training in leadership skills and 
are given the opportunity to parti- 
cipate in the planning and implemen- 
tation of school improvement proj- 
ects. Each team member gains 
management experience in assessing 
needs, stating objectives, planning 
and implementing strategies, and 
assessing progress toward goal s. 
The four teams are the Curriculum 
Support Team, Student Concerns Sup- 
port Team, Student Leadership Team, 
and the Parent Leadership Team. 

Business/Education Partnerships 
(high schools only) are formed 
between the school administration 
and local businesses. Businesses 
offer management assistance and 
other resources to the schools. 

Faculty-Administration Team 

Building includes strategies aimed 
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at involving administrators and 
teachers in school improvement 
activities, and providing training 
for school staff in areas identified 
as weak. 

School-Wide Academic Innovations 

Some academic services that had 
previously been defined as school- 
wide services, such as specialist 
tutoring, "outside" and peer tutor- 
ing, and diagnostic testing/ indivi- 
dualized instruction were focused 
primarily on target students during 
1981-82. Two other services, the 
middle school Exploratory program 
and the Curriculum Guide development 
and distribution component were 
taken over by the school district. 
PATHE continued to assist with these 
programs in several schools, but 
they were not major foci of the pro- 
gram. The remaining school-wide 
academic components are Study Skills 
Training, the Reading Experience 
Program, the Test-taking Skills Pro- 
gram, the Field Trip Program, and 
Student Team Learning. 

Student Team Learning is a set of 
instructional techniques that trans- 
lates into classroom practice sev- 
eral well-known principles of learn- 
ing. Team members in STL classrooms 
study and drill together and prepare 
for quizzes or cross-team competi- 
tions. Rewards for academic 
improvement are given to the hetero- 
geneous student teams based upon the 
progress of the team. The tech- 
niques have received positive evalu- 
ations for enhancing learning, self- 
concept , liking of school and 
increasing cross-race and cross-sex 
friendships (Slavin, in press). 
During the 1981-82 school year spe- 
cial ists in all PATHE schools were 
trained in STL methods , as were 
teachers in four of the seven 
schools. Teachers in three schools 
tried out the methods in their 
classrooms before the close of the 



school year. Additional teacher 
training and careful monitoring of 
the use of the techniques in class- 
rooms are planned for the 1982-83 
school year. 

School-Wide Affective Innovations 

These interventions are targeted 
at general school climate improve- 
ment. Staff counsel ing, previously 
considered a school-wide interven- 
tion, is now being directed primar- 
ily toward target students. The 
subcomponents of the school-wide 
affective intervention are the 
S chool Pr ide Campa ign , which 
includes activities aimed at improv- 
•i ing the overall image of the school, 
and the Extra-Curricular Activities 
program and the Peer Counseling or 
"Rap Sessions" program. 

"Rap sessions" are sessions in 
which adul ts present information on 
topics of concern (such as drug use 
or teenage pregnancy ) to students 
and students react to and discuss 
the topic. These sessions were 
found by the specialists ai^d program 
managers to be an acceptable substi- 
tute for peer counsel ing sessions, 
which had proven difficult to imple- 
ment. 

Career-Oriented Innovations 

Programs to increase transitions 
to college and work (high schools 
only) include the Career Exploration 
Programs — FACET and the Trident Tech 
Fall and Spring Programs, opened to 
women dur ing th e 19 81-82 school 
year — and the Job-seeking Skil Is 
Program. 

Services to Target Students 

This component was strengthened 
in Spring, 1982. The following 
paragraphs describe the redesigned 
program. The actual services pro- 
vided to target students during the 
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1981-82 school were much less 
intensive and less programmatic than 
is implied by the following discus- 
sion. The implementation section of 
this report will discuss in detail 
actual services to target students 
during the 1981-82 year. 

The target student service compo- 
nent is intended to provide affec- 
tive and academic services to a sub- 
population of "target" students 
identified as in need of special 
services. The idea behind this 
strategy is that each school has a 
group of students that will not be 
reached by the organizational -level 
PATHE components. These students 
need highly individualized treatment 
in order to get them to the point 
that they will benefit from the 
classroom and school level PATHE 
innovations. The direct service 
component is intended to identify 
appropriate targe t students, diag- 
nose their needs, define behavioral 
treatment objectives, prescribe 
appropriate intensive services, 
monitor progress toward objectives 
and frequently reassess the appro- 
priateness of the treatment for 
addressing the problem. 

Program Management and Evaluation 

The program managers implemented 
a true experiment , devel oped a data 
base for management information, and 
establ ished implementation standards 
for each program component during 
the 1981-82 year. Each of these 
activities enhanced the manageabil- 
ity and the evaluatabil ity of the 
program. 

Evaluation design . Students eli- 
gible for PATHE direct service were 
randomly assigned to target and con- 
trol conditions in September, 1982 
from a pool of eligibles made up of 
all students in each PATHE school 
and the sixth grades of each feeder 
elementary school who had been sus- 



pended during the 1980-81 school 
year, were referred to the PATHE 
program by a teacher, or whose 
achievement test score (the CTBS 
total test score) fell in the bottom 
twenty-fifth percent of scores for 
that school. Teacher referrals were 
obtained by sending the form shown 
in Table 1 to every homeroom teacher 
in grades five through eleven of the 
PATHE and feeder schools. The cri- 
teria listed at the bottom of the 
form are the same selection criteria 
that were establ ished for the pro- 
gram at the time of the initial 
grant proposal . 

This procedure resulted in pools 
" of different sizes for the seven 
PATHE schools. The pools for the 
high schools, but not the middle 
schools, were of sufficient size to 
constitute target and control groups 
of equal size. In the worst middle 
school case, 71% of the pool had to 
be randomly- assigned - to- the -treat- - - 
ment condition in order to fill the 
100 treatment slots required as a 
condition of the grant. 

Tables 2 "and 3 show the results 
of a post-randomization check. Only 
one of the comparisons shows a sig- 
nificant pre- treatment difference 
between target and control groups. 
At least one significant difference 
at the p<.05 level would be expected 
to occur by chance if the groups 
were, in fact, equivalent. 

Although implementation data to 
be presented later show a partial 
breakdown of the experiment in some 
schools, the design was implemented 
with enough integrity to enable the 
detection of program effects. The 
program managers have taken steps to 
monitor the design of the experiment 
during the 82-83 school year and 
have established clearer expecta- 
tions for level and quality of ser- 
vices to target students. These 
program changes, coupled with the 
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decision to retain the 81-82 target 
and control groups, should result in 
more dramatic treatment-control dif- 
ferences at the end of the 1982-83 
school year. 

Management information system . 
The evaluator established a manage- 
ment information system which organ- 
ized data on level of services to 
all students in the school. Data 
collected in this way were used to 
report on implementation levels, dur- 
ing the Fall '82 semester and again 
for the entire year ( included in 
this report ) . The mid-year report 
based on these data exposed unex- 
pected low levels of direct service 
to target students. It precipitated 
a maj or restructuring of the direct 
service component of the program to 
allow for more intensive and pro- 
grammatic target student services. 

Implementation standards . The 
third major improvement in the man- 
agement of the project was the 
establishment of implementation 
standards for each program compo- 
nent. The program implement ers 
defined the standards and used them 
to monitor the level of implementa- 
tion of each program component. The 
ratings derived from these standards 
will be discussed in the implementa- 
tion section of this report. 

Effectiveness 

The effectiveness of the PATHE 
program is assessed at two levels : 
Comparisons of 1980-81 to 1981-82 
school year school averages and com- 
parisons of randomly assigned treat- 
ment and control students on meas- 
ures of PATHE goals and objectives. 

Measures are taken from the SAES 
student and teacher surveys (Gott- 
fredson, Ogawa, Rickert, & Gottfred- 
son, 1982; Gottf redson, Gottf redson, 
& Cook, 1983) and Charleston County 
School District records of grades, 
California Test of Basic Skills 
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scores, attendance, suspension, 
expulsion, and disciplinary inci- 
dents. The item content and psycho- 
metric properties of all survey 
measures are described in Chapter 3 
of this volume. 

Change from 1981 to 1982 was 
assessed for all outcomes measured 
by th e SAES wi th th e exce p t i on of 
delinquency for the middle schools 
(the Charleston project had the 
self-report delinquency items 
deleted from the Springy 1981 survey 
for their middle school students). 
Bar charts showing the amount of 
change on each outcome for each 
school are available (Gavurin, Hybl 
'»and Gottf redson, 1982) and have been 
used as a formative evaluation tool 
by the PATHE project implementers. 
This report discusses only outcomes 
directly tied to project PATHE' s 
goals and objectives. 

Tables 4a through 4c show differ- 
ences between the 1982 and the 1981 
school scores on survey measures 
relevant to the project's goals and 
obj ectives. Tables 5 and 6 show 
change from basel ine to 1982 on 
measures of goals and objectives for 
which we have data from official 
school records. Tables 7 through 10 
show means for target and random-e- 
quivalent control students on all 
goals and objectives. The discus- 
sion which follows will summarize 
the information in these tables. 

Differences from 1981 to 1982 and 
between the target and control stu- 
dents are generally not large. 
Although few reach statistical sig- 
nificance according to conventional 
standards , some consistent trends 
warrant discussion. Throughout the 
following pages I will consider the 
direction of the observed difference 
as well as the probabil ity of 
observing such a difference if proj- 
ect PATHE is not effective. 
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^-statistics for comparisons of 
target and control groups are based 
on the observed means and standard 
deviations for each group on Spring, 
1982 measures of goals and objec- 
tives, ^.^statistics for comparisons 
of 1981 and 1982 student composi - 
tional measures (i.e., school aver- 
ages on characteristics of the stu- 
dents) are based on the observed 
school means and standard deviations 
for each year . For comparisons of 
1981 and 1982 school climate meas- 
ures (i.e., student and teacher 
reports of the school environment), 
the "t,-statistic" is the ratio of 
the difference between 1982 and 1981 
scores to the standard error of 
measurement of the 1982 scores for 
all but St. Paul schools in the 
OJJDP initiative. Climate measure 
differences that exceed twice their 
standard error of measurement may be 
regarded as dependable. 

The following paragraphs use 
information about the direction of 
the difference score to assess 
school level effects of the PATHE 
program. The reader should be aware 
that some of the differences are 
close to zero, but are still inter- 
preted as improvements or regres- 
sions, depending on the sign. 

Delinquenc_y 

School level . Measures of delin- 
quent or disruptive behavior — sus- 
pensions and self-reported delin- 
quency — show the same pattern of 
resul ts for high schools (Tables 4c 
and 5) . Each of the three PATHE 
high school s experienced a decl ine 
from 1980-81 to 1981-82 in both the 
number of suspensions and self-re- 
ported delinquent behavior. For St. 
John's high school, the reductions 
in Drug Use and Total Delinquency 
were statistically significant. For 
the other two high schools, the 
decline in suspensions was statisti- 



cally significant. The high school 
control school experienced an 
increase in suspensions and in 
self-reported delinquent behavior. 

Progress in the middle schools on 
this outcome is more difficult to 
assess: The self-report delinquency 
items were deleted from the Spring, 
1981 middle school survey , and 
aggregate level suspension data from 
official records (Table 5) do not 
accord with sel f-reported suspen- 
sions from the survey . (Table 4a). 
Official records show an increase in 
suspensions for three of the four 
PAIHE middle schools as well as 
their comparison school. Self-re- 
ports suggest that suspensions were 
down in two of four PATHE middle 
schools as well as in the comparison 
middle school. 

Alternative measures of school 
disruption shown in Table 4c show an 
increase in- students'- and teachers' 
perception of safety and a decrease 
in student reports of victimization 
in six of the seven PATHE schools. 
Several of these changes were sta- 
tistically significant. Both com- 
parison schools experienced 
increased student victimization but 
students in one and teachers in both 
comparison schools reported more 
school safety. 

Individual-level . Treatment-con- 
trol student comparisons on measures 
of delinquency (Table 7) show little 
evidence of a program effect on this 
outcome. Although one of the seven 
schools showed a sig. if icant reduc- 
tion in delinquency among target 
students, the direction of the 
treatment-control difference favored 
control students about as of ten as 
target students in the other 
schools. Comparisons of suspensions 
and disciplinary incidents in the 
school show similar results. 
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Attendance 

School-level > Attendance rates 
in the Charleston County schools are 
typically high (in 1980 the average 
daily attendance rates for the PATHE 
and control schools ranged from 89 
to 94 percent). Ceiling effects, 
the initiation during the 1981-82 
school year of a st rict district- 
wide attendance policy, and the fact 
that most school personnel (includ- 
ing PATHE special ists) do not per- 
ceive nonattendance to be a problem, 
all reduce the likelihood of program 
effects on attendance rates. Tables 
4c and 5 show that attendance did in 
fact increase in all three PATHE 
high schools and one or two of the 
PATHE middle schools (depending on 
the source ) • 

Individual level . The target- 
control comparisons are prcxnising 
for both high and middle PATHS 
schools (Table 8): Although the 
differences never reach statistical 
significance, target students in 
five of the seven schools have a 
lower proportion of unexcused days 
absent to total days enrolled than 
do control students. The other two 
school s show no difference between 
target and control students. 

Academic Achievement 

School-level . All Charleston 
schools, PATHE and control, improved 
from 1981 to 1982 on California Test 
of Basic Skills scores, and this 
improvement was more pronounced in 
the middle than in the high schools 
(see Table 6) . The extent to which 
project PATHE contributed to the 
improvement is unclear, but the evi- 
dence favors PATHK: By and large 
the improvement in Charleston high 
schools on CTBS test scores was min- 
imal and did not reach statistical 
significance. One PATHE high 

school — Burke — was a clear exception 
to th is rul e . Tabl e 6 shows a con- 
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sis tent statistical ly significant 
increase in test scores for Burke . 
Evidence from many sources — ratings 
of the strength and f idel ity of 
implementation and observations and 
interviews with the Burke 
specialists — show that Burke imple- 
mented the school-wide academic com- 
ponents of the program with much 
more f idel ity to the program plan 
than did the other PATHE high 
schools. 

Individual-level . ' Other evidence 
f av or ing a po s i t iv e ef f ec t of the 
PATHE ' program on achievement comes 
from target-control comparisons 
(Table 9). Comparisons of the per- 
cent scoring in the bottom national 
quartile of the 1982 CTBS show that 
in only one school did the control 
students fare better than the target 
students. The overall treatment- 
control difference reaches statisti- 
cal significance in Language, an 
area that several PATHE specialists 
focused on during the year, as well 
as for the total battery. The 
direction of the difference favors 
the target students for all subar- 
eas. Target-control comparisons of 
school grades al so favor PATHE in 
six of the seven schools, and the 
difference for percent receiving 
fail ing grades averaged across al 1 
subject areas reaches statistical 
significance. 

Transition to Employment and Post- 
secondary Education 

This outcome will be assessed in 
the final report, but see the "Edu- 
cational Expectations" objective 
below. 

PATHE Obiectives 

The PATHE managers identified a 
number of intermediary outcomes 
toward which the program is aimed. 
These intermediary outcomes, or 
"obj ectives, " are primarily student 



230 



PATHE 



and teacher attitudes, but also 
include certain teacher behaviors. 
Relevant comparisons for each of the 
PATHE objectives are shown in Tables 
4a, 4b, 4c, and 10. Because many of 
the objective measures are somewhat 
redundant, I will summarize only the 
major dimensions here. 

Teacher Morale (Teacher Reports) 

Every PATHE school and the middle 
school comparison school experienced 
an increase in teacher morale. The 
high school comparison school 
declined in teacher morale. 

Reports^ of Planning & Action and 
Smooth administration 

Both teachers and students 
reported on the extent to which 
their school is characterized by 
planfulness and innovative atti- 
tudes, and teachers reported on 
their perceptions of the extent to 
which the school is run smoothly and 
effectively. According to students 
in four of the seven PATHE schools, 
the degree of planning decreased 
from 1981 to 1982. Three of the 
four schools that did not improve 
were al ready above the 85 th percen- 
tile for schools in the study, 
though. Students in both comparison 
schools reported a decline in plan- 
ning. 

Teachers reported an increase in 
pi anning and in smoo th adminis tr a- 
tion in five of the PATHE school s. 
Both comparisons schools declined on 
both of these teacher measures. 

Teacher Expectations for Student 
Achievement (Teacher Reports) 

The direction of change was posi- 
tive in f o ur of th e sev en PATH E 
school s — two middle and two 
high — and in one of the two control 
schools. 



Student Reports of Rewards and Pun - 
ishments in School 

Students in five PATHE schools 
reported an increase in rewarding 
and a decrease in punishing experi- 
ences in school from 1981 to 1982. 
Both comparison schools experienced 
a decrease in these intermediary 
outcomes. Of the two PATHE middle 
schools with increases in students^ 
repor ts of school experiences as 
punishing, one was already extremely 
low on this measure. 

Treatment/control comparisons of 
rewarding and punishing experiences 
in the middle schools showed differ- 
'■*ences favoring the treatment stu- 
dents in three of the four schools. 
One comparison reached statistical 
significance . In only one of the 
three high schools did the direction 
of the treatment/ control difference 
favor the treatment students. 

Students Reports of Fairness and 
Clarity of Rules 

Three of the PATHE school s 
increased on both measures, and 
another two increased on students^ 
perceptions of the clarity of the 
rules, though not the fairness. Two 
of the seven PATHE school s decl ined 
on bo th mea s ur e s , and bo th compa r i- 
son schools declined on both meas- 
ures. The magnitude of the changes 
from year to year is noteworthy . 
The two PATHE schools that declined 
from 1981 to 1982 on the Clarity of 
Rul es scale decl ined onl y si igh tl y 
( in fact , the difference score 
equals zero when rounded off). The 
decl ine experienced in the compari- 
son schools was greater. The major 
PATHE activities aimed at increasing 
rule fairness and clarity occurred 
during the 1980-81 school year. The 
PATHE discipline referral system was 
instituted during that year, and 
several PATHE schools had school- 
wide campaigns to generate and pub- 
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Attachment to School 

The program had positive effects 
on this key intermediary outcome in 
the middle schools. All improved on 
this outcome while the comparison 
middle school declined substan- 
tially. Only one of the PATHE high 
schools improved on this outcome, 
but of the two that declined one 
started out and remained above the 
94th percentile. Middle school tar- 
get students reported higher levels 
of attachment in every school, but 
only in one high school did this 
comparison favor the program (in a 
second high school . there v/as no dif- 
ference). The high school with the 
highest school-wide attachment had 
an apparent negative effect on its 
target students (p<.01). 

Educational Expectations 

Students in two middle schools 
and two high schools in the PATHE 
program reported increased educa- 
tional expectations. Students in 
one comparison school also reported 
increased expectations. In only one 
treatment/control comparison did the 
direction of the difference favor 
the PATHE target students. 

Belief in Rules 

All but one of the Charleston 
schools (a PATHE school) increased 
on this measure. Roughly half of 
the treatment/control comparisons 
favored the PATHE students. 

A Closer Look at Burke 

The most surprising finding of 
this interim evaluation is a consis- 
tent trend toward a negative effect 
on target students in Burke, one of 
the PATHE high schools. The result 
is doubly troubling because the 
Burke operation is regarded as the 
closest approximation (among high 
schools) to the ideal implementation 
of the PATHE program. The school 



stands out on all measures of imple- 
mentation for school-wide activi- 
ties, and its improvement from 1981 
to 1982 on PATHE goals and objec- 
tives was second only to Haut Gap, 
the middle school which al so imple- 
mented the program with excellence. 
How might Burke's negative results 
be explained? There are several 
possibil ities : 

1) The control students received 
more treatment than the tar- 
get students, 

2) The "worst" control students 
dropped out of school while 

* their equivalent target stu- 
dents remained in school , 
thus making the groups none- 
quivalent (because dropouts 
were not included in the data 
collection), or the "worst" 
control students did not take 
the spring 1982 survey, and 

3) The treatment was harmful. 
Each will be discussed in turn. 

Table 11 compares the total num- 
ber of contacts with specialists 
recorded for the 1981-82 school year 
for Burke High School and all other 
PATHE schools. It shows that con- 
trol students did not receive more 
services than treatment students at. 
Burke. This is true also for the 
subset of target and control stu- 
dents who took the Spring survey. 
The PATHE special ists in the school 
admitted that they may have been 
more careful to record contacts with 
target than control students because 
they were concerned with "getting 
credit for" their target student 
contacts. The underreporting of 
control student contacts would have 
had to have been blatant in order to 
explain the differences on outcomes 
favoring control students, though. 

The second hypothesis , that of 
differential attrition from school 
or from the survey administration, 
is also not supported by the data. 
The "original" and "final" columns 
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in Table 2 show no pre-treatment 
difference between CTBS scores for 
target and control students who left 
school before Spring, 1982 when the 
CTBS test was administered. Accord- 
ing to these figures, dropout did 
not change the equivalency of the 
target and control groups. Similar 
analyses comparing 1981 CTBS meas- 
ures for target and control students 
who took the Spring, 1982 survey 
also disconfirmed the differential 
attrition hypothesis. Among stu- 
dents who took the 1982 survey at 
Burke, control students did not have 
a pre-treatment "advantage" over 
target students. 

Might the treatment at Burke have 
caused target students to report 
less attachment to school, score 
lower on achievement tests, have 
lower expectations for completing 
high school and have fewer jobs than 
equivalent students who received one 
third as much treatment? Observa- 
tions of the Burke specialists in 
action imply that the opposite 
should occur. Both specialists are 
experienced teachers, have excellent 
rapport with the students, teachers 
and administrators in the school, 
are perceived as extremely hard- 
working individuals, are advocates 
for the students while being firm 
disciplinarians, and above all are 
dedicated to their work. Their 
teaching style seems to accord with 
that suggested by research: Their 
instructional pace is rapid and 
their tutoring group size is always 
sufficiently small to allow for 
individualized attention. 

Two hypotheses about the way the 
PATHE treatment was administered at 
Burke were examined: Differences 
between Burke and other schools in 
the standard deviation for number of 
contacts (see Table 11 ) raises sus- 
picion. The frequency distributions 
for total number of contacts show 
that at Burke a few students (both 
treatment and control) had extremely 



high numbers of contacts v;ith the 
specialists. Although the distribu- 
tions for most of the students in 
Burke and other schools are similar, 
the top 5% of the distribution 
received between 45 and 60 contacts 
at Burke and only between 18 and 45 
contacts at the other schools. Tar- 
get students were more likely to be 
among those students receiving 
extreme amounts of service . The 
hypothesis that a high amount of 
contact with specialists causes tar- 
get students to disl ike school was 
examined and disconf irmed. Number 
of contacts with special ists is 
si ightly positively related to 
attachment to school for target stu- 
' dents (r=.07, p=.08) and slightly 
negatively related for control stu- 
dents (r=-.09, p=.06) for all PATHE 
schools. The associations are 
stronger at Burke, reaching statis- 
tical significance at the p<.05 
level (target: r=.21; control: 
r=-.28). Target students who 
received extreme amounts of service 
liked school more , not less, than 
those receiving less service. 

One final difference between the 
way direct services are provided at 
Burke and other school s is notewor- 
thy. Tables 15 through 21 show that 
Burke is the only PATHE school which 
provided more academic than affec- 
tive types of services to target 
students. The difference may have 
resulted from differences in record- 
ing practices across schools, but 
observations and interviews with the 
special ist at Burke also suggest 
that the emphasis is on academics. 
The Burke specialists frequently 
work as a team with small groups of 
students, tutoring and providing 
counsel ing as needed. Tutoring 
seems to be the major focus in these 
smal 1 group sessions . PATHE spe- 
cial ists at other schools work more 
independently and schedule entire 
sessions solely for counseling. 
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The hypothesis that the ratio of 
academic to affective services is 
central to the success of the PATHE 
program is currently being tested. 
This ratio is a variable central to 
the PATHE philosophy: The program 
rests on the notion that students 
need both academic and affective 
support. The program managers chose 
to compromise this notion when they 
responded to the federal budget cut 
by severely curtailing affective 
services in half of the PATHE 
schools for the 1982-83 school year. 

Summary of Outcome Evaluation 

The effects of the PATHE program 
differ according to target (i.e., 
treatment-control comparisons vs. 
school-level change) and level 
(i.e., middle vs. high school s ) . 

Target Students 

Program effects on target stu- 
dents are more evident at the middle 
than at the high school level. In 
general, target students in the 
PATHE high schools seem not to have 
benefited from the program any more 
than did the typical, nontargeted 
student in the school during the 
1981-82 school year. The results 
for academic achievement are an 
exception, though: Target students 
in all PATHE schools save one high 
school fared better than their ran- 
dom-equivalent controls on this out- 
come. Target students in all middle 
school s attended school more often 
and out-achieved the controls, and 
they reported higher levels of 
school attachment and more positive 
self-concepts. Only for the 

achievement outcomes do the differ- 
ences reach statistical signifi- 
cance, though. 



School-level 

Comparisons of 1981 to 1982 
changes in school averages on 
PATHE's goals and objectives must be 
approached with more skepticism than 
comparisons of randomly assigned 
treatment and control students. 
Even when improvement is observed 
from year to year we cannot be sure 
that the PATHE program caused the 
improvement. Schools were not ran- 
domly assigned to treatment , and we 
have only two Charleston school s 
with .which to compare the PATHE 
school s. 

School-wide improvements on PATHE 
goals and objectives are generally 
more evident at the high than at the 
middle school level, although the 
"most improved" school is a middle 
school. PATHE high schools improved 
on 80% of the outcomes examined, 
while the PATHE middle schools 
improved on 69%.'' "All ~ three*" PATHE 
high schools and two of the four 
PATHE middle schools improved si.^ni- 
ficantly on several of the outcomes 
examined in Tables 4a and 4b. 

The Sign Test (Siegal, 1956) can 
be appl ied to the results in Table 
4c to determine the probability that 
the observed pattern of changes 
could have resulted by chance. The 
direction of 1981-1982 change is 
treated as a binomial variable with 
a probabil ity of improving of .5 . 
Several of the findings discussed 
above are found to be improbable 
when this test is applied. For 
exampl e , the observed deer ease in 
student al ienation in every PATHE 
school and increases in teacher 
morale and student positive self- 
concept in every PATHE school would 
occur by chance only eight times in 
one thousand. The patterns of 
increases and decreases for stu- 
dents' and teachers' reports of 
safety , student victimization and 
bel ief have only a . 06 probabil ity 
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of occurring by chance. The PATHE 
school improvements for teacher 
reports of smooth administration and 
planning and action, student reports 
of rewarding school experiences, 
clarity of rules, interpersonal com- 
petency, and attachment to school 
also have a low likelihood of occur- 
ring by chance given that both com~ 
parison school s regressed on these 
measures. All of the above-men- 
tioned findings favor the PATHE pro- 
gram. None of the variables exam- 
ined in Tables 4a through 4c show 
disadvantages for the PATHE schools. 
These probabil ities and those 
reported in the following paragraph 
are not corrected for possible ceil- 
ing effects which, as I mentioned in 
the previous section, sometimes make 
improvement from 1981 to 1982 impro- 
bable. 

A gross measure of program effec- 
tiveness is simply the ratio of out- 
comes on which a given school 
improved from 1981 to 1982. If we 
assign a score of one to each out- 
come examined in Tables 4a through 
4c that moved in the positive direc- 
tion and a score of zero to those 
that did not, and then rank the 
seven program and two comparison 
schools from lowest to highest on 
the degree of positive change from 
1981 to 1982 there is no overlap 
between the average rankings for the 
comparison and the PATHE schools. 
That is , the average improvement for 
the highest-ranking comparison 
school is lower than the average 
improvement for the lowest-ranking 
PATHE school . PATHE school s 

improved on 64% to 82% of the out- 
comes in Tabl es 4a through 4c with 
an average of 74% improvement. The 
comparison school s improved on 24% 
and 45 % of th e out come s with an 
average of 34%. 



Program Implementation 

Information on level of program 
implementation comes from two main 
sources. "Intensity" standards , 
(standards for the amount of ser- 
vices provided by PATHE personnel), 
and "f idel ity" standards (standards 
for the quality of the services ren- 
dered) were developed by the program 
managers to assist in project man- 
agement and evaluation. Each school 
in the PATHE program was rated for 
intensity and fidelity -twice during 
the 81-82 school year, once at mid- 
year and again at the end of the 
school year. A five point scale 
ranging from zero (does not meet 
standard/not implemented) to 4 
(exceeds standard) was used. The 
ratings were based upon documented 
contacts with students, teachers and 
parents, minutes and agendas from 
meetings, logs of resources used and 
interviews with PATHE special ists. 
The standards 'are" shown in Tables "12 
and 13, and the averages of the two 
ratings for each school are shown in 
Table 14. 

The second major source of pro- 
gram implementation data for 1981-82 
is information about the nature and 
duration of PATHE specialist-student 
contacts. These data come from a 
management information system devel- 
oped by Ann Birdsey e, the former 
PATHE evaluator. Data from contact 
logs and sign-in sheets were entered 
into computer files and reorganized 
to facil it ate analysis. Tables 15 
through 21 compare levels of treat- 
ment for target and control students 
and for the entire school. Several 
points can be drawn from these 
tables: 

Experimental design breakdown . 
The experimental design was not 
upheld in most school s. The most 
severe breakdowns occured in Courte- 
nay and Rivers middle schools, where 
control students received close to 
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one half as much treatment as target 
students. In both of these schools, 
control students received more 
direct services than did other non- 
targeted students in the school. In 
other words, it appears that control 
students were singled out for treat- 
ment or that the specialists in 
these school s had special difficul- 
ties withholding treatment from con- 
trol students referred for service. 
Control students in Burke al so 
received more treatment than other 
non-targeted students, and received 
about one third as much direct ser- 
vice contact as the target students. 
All other schools except for Haut 
Gap treated control students one- 
fifth to one-third as much as the 
target students, but these ratios 
would be expected if control stu- 
dents received only those services 
directed at all students in the 
school, which was in accord with the 
experimental design. 

Intensity of direct services . 
The level of direct services to tar- 
get students did not reach accepta- 
ble levels in most schools. Accord- 
ing to the intensity standards set 
by the program managers, target stu- 
dents were to have at least eight 
contacts with the specialists during 
the year. Rhett exceeded this stan- 
dard but was the only PATHE school 
to meet it. Haut Gap and Brown fell 
seriously short with less than half 
of the required number of contacts. 

Overall program accomplishments . 
On the average, PATHE schools met 
about 75% of the program standards. 
The academic interventions were 
implemented most faithfully and the 
services to target students least 
faithfully program-wide. The inten- 
sity and f idel ity ratings suggest 
that the team structure and the 
career interventions were imple- 
mented almost up to standard in the 
high schools, and that the affective 



interventions lagged behind. 
According to the ratings, the middle 
schools implemented the program with 
less fidelity than the high schools. 

The most vigorously implemented 
components include Student Leader- 
ship Teams and Curriculum Review and 
Revision for all schools, and Parent 
Leadership Teams , School Pride Cam- 
paigns, Field Trips, Reading Experi- 
ence Program, Job-Seeking Skills and 
Faculty Inservices for high schools 
only. The least well-implemented 
components included services to tar- 
get students and extracurricular 
activities (all schools ) and Busi- 
ness-Education Partnerships, Peer 
Counseling, Tutoring, Job-Seeking 
Skills and Faculty Inservices (mid- 
dle schools only). 

Caution should be exercised in 
interpreting the implementation 
data. Only a moderate degree of 
correspondence between outcome meas- 
'ires and implementation levels was 
cL served: School -wide academic and 
affective components were imple- 
mented bes t , and the outcome data 
show most improvement in these 
areas. By and large , however, the 
schools which would be expected, on 
the basis of the implementation 
data, to show the most improvement 
in certain areas often did not. The 
moderate level of correspondence is 
explained by two factors: The proj- 
ect never established a workable 
system for collecting data on some 
PATHE activities — notably extracur- 
ricular activities and school-wide 
activities such as the study and 
test-taking skills programs. Also, 
the intensity and fidelity standards 
were sometimes unreal istic and were 
not well-understood by the PATHE 
special ists . This problem was cor- 
rected for the 1982-83 year. The 
standards are now more realistic and 
have been communicated to all spe- 
cial ists. 
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Discussion 

The establishment of an experi- 
mental design, implementation stan- 
dards and a system for monitoring 
program activities during the 
1981-S2 school year strengthened the 
PAIHE program and its evaluation. 
The program was more focused and 
systematically implemented than in 
its previous year. Specialists 
reported that the monitoring system 
f acil itated their jobs because it 
helped them to organize and priori- 
tize activities and to manage their 
time better. 

Educational expectations did not 
improve . One objective central to 
the PATHE program was not met during 
the 1981-82 year. The expectations 
of students and teachers for the 
students^ academic success did not 
increase. In 1981, teachers in 
PATHE schools reported that between 
28 and 39% of the students they 
teach are low ability students. 
Between 55 and 77% of students in 
these same schools reported that 
they expected to continue some type 
of schooling after high school. The 
1982 figures are roughly the same. 
Target-control student comparisons 
on this outcome also tended toward 
the negative: Control students in 
five of the seven PATHE schools had 
higher educational expectations than 
did the target students, and the 
difference reached statistical sig- 
nificance in one school. 

Although the largest school -wide 
gain in this area was observed in 
Burke — the school which implemented 
the school-level components of the 
the program with most fidelity to 
the PATHE theory, the absence of 
evidence of overall program effec- 
tiveness in this area suggests the 
need for better-defined program com- 
ponents aimed at increasing expecta- 
tions. The "individual ized" 
approach taken by many educators and 
the rel iance of many counsel ing 



techniques on information about the 
child^s background may in fact serve 
to foster low expectations. 

An example of subtle undermining 
of the program objective comes from 
an interview with one PATHE special- 
ist who reported frequently taking 
teachers aside and "filling them in" 
about the child^s home life so that 
the teacher might react more appro- 
priately to the child's behavior. 
Although offering . this type of 
"assistance" is probably not an 
uncommon pr act i ce among counsel or s , 
it certainly is not in accord with 
the PATHE model. 

' Projected Program Improvements 

The program managers have already 
taken steps to improve the program 
in areas identified as weak during 
the 1981-82 school year. The most 
dramatic change is in the target 
student' service "component*. ^ Blue- 
prints for serv ice s to targe t stu- 
dents have been developed and shared 
with the specialists, and these ser- 
vices are monitored carefully by the 
program manager and evaluator on an 
ongoing basis. Early signs from the 
first quarter of the 1982-83 school 
year suggest that target students 
are receiving much more service of a 
higher quality than they were last 
year. This improvement also seems 
to be solving the problem of the 
experimental design breakdown: The 
increased expectations for services 
to target students made it necessary 
for specialists to establish treat- 
ment schedules for students. In 
order to keep up with the schedules, 
specialists are withholding services 
to nontarget students more often. 

An improved system for monitoring 
program activities was developed and 
implemented in Fall, 1982. The new 
system identifies problem areas 
early so that the program . manager 
can work with the special ists to 
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correct implementation breakdowns in 
a timely fashion. Early evidence 
also suggests that this system is 
strengthening the PATHE program. 

Conclusion 

Interim results of the evaluation 
of Project PATHE suggest the follow- 
ing: 

1 . Program effects on high risk 
students targeted by the PATHE pro- 
gram are more evident at the middle 
than at the high school level. 

2 . Target students in all middl e 
and two high schools scored higher 
on standardized achievement tests 
and received higher grades than did 
random control students. These 
results reached statistical signifi- 
cance. 

3. Target students in all four 
middle schools attended school more 
often, reported higher levels of 
attachment to .<^chool , and more posi- 
tive self-concepts than did the con- 
trols, although' these differences 
did not reach statistical signifi- 
cance. 

4 . Target sf *dents in one middl e 
school reported significantly less 
serious delinquent behavior than 
random-equivalent control students. 

" . School -V .de improvements on 



PATHE goals and objectives are gen- 
erally more evident at the high than 
at the middle school level. PATHE 
high schools improved from 1981-1982 
on 80% of the outcomes examined. 
PATHE middle school improved on 69% 
on the average. 

6 . Every PATHE school improved 
from 1981-1982 on student Alienation 
and Positive Self -Concept and 
Teacher Morale. Six of the seven 
PATHE schools improved on students^ 
reports of Victimization, and 
Safety., Belief in Rules, and 
teachers^ reports of Safety. 

7. Every PATHE high school exper- 
ienced a reduction in delinquent 
behavior and suspensions from 
1981-1982. For one the reduction in 
Delinquency was statistically signi- 
ficant , and for the other two the 
reduction in the number of suspen- 
sions was statistically significant. 

8. School-wide academic interven- 
tions were implemented with most 
integrity during the 1981-82 year. 
The high schools generally met or 
exceeded program standards in this 
area, and the middle school is met 
about 75% of the standards. 

9. Target student services were 
generally weak, compared to program 
standards. Only one middle school 
met the intensi ty standards estab- 
lished by the program managers. 
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I I I I I I I . I I I I 

♦ 4. 4. 4.- 4. 4- 4. 4. 4. 4. 

I I I I * I I I I I I I 

♦ 4. 4. 4. 4. ♦ ♦ -4. 4. 4^-. 

I I I I . I I I I I I I 
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I ' I I I I I I I I I I 

4 , 4. 4. 4. ♦ 4. 4. 4. 4. 4. 4.— 

I I I . I • I I I I I I I 

♦ 4. + 4. 4. 4. 4. 4. 4. 4. 4.— 

^Iciso list ony other 3tu.iF:nts not in /our homeroom who-Should he selected as PATl 

t.jr ;ot stu:!-jnts next y-^oir: 
♦ 4.-- 4. 4.---^ 4. ♦ 4. 4. 4. ♦ 4.-- 

I II I '1 I I I I ! I 
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I III I I I I I I I 

I I c I I I I I I I I 

I ' I I I I I I I I I i 

♦ 4. «• 7 + ♦•-^ 

I I I I I I ' I I I .1 I 

^ 4. 4. 4- «• ^ ^ 

<^^'u.'Pbers one throu gh seven refer t o the *f ol 1 ow i n j PATHEts^l ect ion cr i ter ia: ^ 

I. previous exDulsionf suspension? or involveiment in a number of discioVinary 

actions . 
2» social promotion or pr ev i ou s retent i on 

3. decline in performance level over past 3 school, years 

4. pr'*3vious request for guidance ar counsel i ng due to a special need or problem; 
tendency to cut class itruant) 

6. difficulty with the .law (delinquent) 

7. referral by a school /commun i ty agency 241 
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Table 2 



Spring 1981 California Test of Basic Skills Standard Scores 
for Target and Control Groups, by School 



Total 



Reading 
I SD N 



Math 



X SD N 



■Language 
X SD N 



Original 
X SD N 



Final 



X SD N 



Courteney 
Target 
Control 



(N=83) 
(N=80) 



376 68 79 
m 62 77 



389 56 
388 53 



78 
76 



m 65 
397 60 



79 
77 



363 68 78 
361 60 76. 



365 68 
362 60 



74 
73 



Rhett 
Target 
Control 



(N=62) 
(N=54) 



415 58 60 
406 64 50 



407 55 61 
405 42 49 



423 65 
420 59 



60 
50 



390 62 60 
388 51 49 



389 63 
389 52 



55 

46 



Rivers 
Target 
Control 



(N=86) 
(N=75) 



400 65 77 
399 70 65 



396 50 79 
400 43 66 



414 59 77 
419 63 66 



380 54 77 
383 56 64 



382 53 68 
382 59 56 



Haut Gap 
Target 
Control 



(N=65) 
(N=23) 



407 69 59 
378 48 21 



401 46 
378 51 



58 
21 



422 60 
392 57 



59 

21 



387* 58 57 
354 53 20 



385 54 
358 47 



51 

18 



Brown 
Target 
Control 



(N=83) 
(N=80) 



449 77 73 

450 68 71 



456 70 
455 66 



70 
70 



466 69 
457 66 



73 
71 



445 75 70 
440 69 70 



448 74 . 58 
439 68 * 61 



Burke 
Target 
Control 



(N=80) 
(N=77) 



442 82 79 
436 81 75 



424 63 
424 62 



74 
71 



444 78 
441 71 



79 
75 



414 69 
414 71 



73 

71 



418 72 61 
420 68 57 



St. Johns 
Target 
Control 

All Schools 
Tar<;et 
Control 



(N=78) 
(N=76) 



(N=540) 
(N=464) 



469 87 65 
464 69 64 



422, 79 492 
419 75 423 



458 78 64 
453 71 62 



418 66 484 
418 63 415 



466 74 66 
454 72 65 



I 



434 71 493 
430 68 425 



450 83 62 
442 '73 60 



403 74 477 
402 71 410 



449 85 
430 69 



43 
48 



402 73 410 
400 69 359 



Difference between target and control groups is significant at the p<.05 level. 

^Figures under the "original" column represent group averages for all 1981-8 2 target and control students who 
took the CTBS test in Spring, 1981. Figures under the "final" column represent averages for all target ai^d 
control students who took the Spring, 1981 CTBS test as well as the Spring, 1982 CTBS test. Total N's (in 24 0 
j?i-irentheses) are the number of cases initially randomized into treatment and control conditions. 



Table 3 



Means and Standard Deviations 
for Teacher Referrals to PATHE 
and Number of Suspensions, 1980-81 



Number of 
Referrals 
to PATHE 

X SD 



Number of 
of 

Suspensions 
X SD 



Courtenay 

Treatment (N=83) 
Control (N=80) 



.61 
.54 



.49 
.50 



.06 
.08 



.24 
.31 



Rhett 



Treatment (N=63) 
Control (N=52) 



.46 
.58 



.50 
.50 



.25 
.17 



.57 
.43 



Rivers 

Treatment (N=85) 
Control (N=75) 



.53 
.63 



.50 
.49 



.25 
.12 



.53 
.40 



Haut Gap 

Treatment (N=64) 
Control (N=23) 



.50 
.39 



,50 
.50 



.12 
.00 



.33 
.00 



Brown 



Treatment (N=81) 
Control (N=78) 



,63 
.50 



.48 
.50 



.63 
.58 



.86 
.75 



Burke 



Treatment (N=86) 
Control (N=80) 



.20 
.18 



.40 
.38 



.24 
.31 



.51 
.61 



St. Johns 

Treatment (N=78) 
Control (N=76) 



.46 
.43 



.50 
.50 



.77 1.02 
.88 1.36 



All Schools 

Treatment (N=3'i0) 
Control (N=464) 



.48 
.46 



.50 
.50 



,34 
.35 



.68 
.78 



Note Total N's (in parentheses) are numbers of cases initially 

randomized into treatment and control conditions. 
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Table 4a 



1981 to 1982 Difference Scores for School Averages 
on Survey Measures of PATHE Goals and Objectives — Middle Schools 



Goal or objective measure 


Courtenay 


Rhett 


Rivers 


Haut 


Gap 


Laing 
(control ) 


diff 


t 


diff 


t 


diff 


t 


diff 


t 


diff 


t 


Total Delinquency (-) 































Serious Delinquency (-) 
Drug Use (-) 




— — 







































— . 


Suspensions (-) 


.01 


.19 


-.08 


-2.62 


.01 


.37 




-.01 


-.18 


-.03 




-.96 


Victimization (-) 


-.03 


-1.61 


-.01 


-.86 


-.02 


-1.33 


.02 


1.69 


.00 


.05 


School Nonattendance (-) 


.02 


.57 


.03 


.67 


.00 


-.00 


-.03 


-1.06 


.01 


.27 


School Rewards (+) 


.02 


.71 


.02 


.60 


-.02 


-.88 


.02 


.65 


-.03 


-1.19 


School Punishments (-) 


-.01 


-.30 


.02 


.98 


.02 


.71 


-.02 


-.87 


.05 


2.45 


Interpersonal Competency (+) 


.02 


1.29 


.05 


2.68 


.02 


1.15 


.03 


1.31 


-.02 


-.88 


Positive Self-concept (+) 


.00 


.26 


.02 


1.04 


.03 


1.69 


.04 


2.18 


.02 


1.52 


Alienation (-) 


- .01 


-.36 


-.07 


-2.97 


-.03 


-1 .21 


-.05 


-l.c3 


.00 


-.15 


School Attachment (+) 


.00 


.05 


.02 


1.17 


.01 


.27 


.04 


1.76 


-.05 


-2.65 


Educational Expectations (+) 


-.06 


-.46 


.01 


.11 


-.14 


-.92 


.29 


1.92 


.10 


.69 


RaI 1 laf ( + ^ 
OkH, i-ci. V ~/ 


.00 


-.11 


.03 


1.39 ' 


.03 


~1.26 


OS' 


' 9"" 9 
^ . «j ^ 


.04 


1.85 


Low Expectations^ (-) 


-4.17 


-.72 


.30 


.04 


.90 


.17 


-5.51 


-.97- 


■10.99 


-1.57 


Involvement (+) 


.02 


1.41 


-.00 


-.10 


-.03 


-1.64 


-.01 


-.49 


.01 


.41 


Victimization^ (-) 


-.01 


-.25 


.00 


.07 


-.01 


-.32 


.02 


.29 


-.00 


-.10 


Safety^ (+) 


.34 


2.28 


.32 


2.17 


-.03 


-.19 


.07 


.49 


.12 


.83 


Teacher Morale^ (+) 


.07 


1.26 


.09 


1.67 


.05 


1.03 


.15 


2.78 


.00 


.07 


Planning and Action^ (+) 


-,02 


-.24 


.06 


.87 


.10 


1.44 


.03 


1.17 


-.07 


-1.01 


Smooth Administration^ (+) 


-.02 


-.34 


-.10 


-1.68 


.06 


1.03 


.15 


2.46 


-.02 


-.31 


Rule Clarity ( + ) 


.00 


-.10 


.01 


.23 


.02 


.51 


.02 


.47 


-.04 


-.90 


Rule Fairness (+) 


-.01 


-.22 


.02 


.37 


.00 


.03 


-.01 


-.10 


-.04 


-.67 


Planning and Action (+) 


-.02 


-.34 


-.05 


-1.01 


.03 


.73 


.00 


.10 


-.06 


-1.30 


Safety (+) 


.04 


1.44 


.03 


1.01 


.05 


1.70 


.00 


.06 


.02 


.76 


Pet. measures improved 


.68 




.f,8 




.64 




.82 




.45 





Note , The desired direction of change is indicated in parentheses after each scale name, 
^-statistics for compositional measures (Total Delinquency through Involvement) are based 
upon 1981 and 1982 means and standard deviations for each school. For psychosocial cli- 
mate measures, the "t.-statistic" is the ratio of the difference between 1982 and 1981 
scores, to the standard error of measurement of the 1982 scores for all non-St. Paul 
schools in the OJJDP initiative. As a rule of thumb, psychosocial climate measure differ- . 
ences that are twice the standard error of measurement may be regarded as dependable. For 
compositional measures, t. values of 1.96 and 2.54 are statistically significant at the 
p<.05 and p<.01 level, respectively. All measures are taken from the SAES student surveys 
unless otherwise indicated. Dashes indicate that the goal was not measured in 1983. 

a 

Measure taken from SAES teacher survey 
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.11 
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.00 
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Table 4c 



Number of PATHE and Control Schools that Improved from 
1981 to 1982 on Survey Measures of PATHP) Goals and Objectives 



PATllE Schools Control Schools 



Middle High Total Middle High Total 
Goal or objective measure (n«4) (n=3) (n»7) (n«=l) (n=l) (n=2) 



Total Delinquency (-) 
Serious Delinquency (-) 
Drug Use (-) 
Suspensions (-) 
Victimization (-) 
School Nonattendance (-) 
School Rewards (+) 
School Punishments (-) 
Interpersonal Competency (+) 
Positive Self-concept (+) 
Alienation (-) 
School Attachment (+) 
Educational Expectations (+) 
Belief (+) 

Low Expectations^ (-) 
Involvement (+) 
Victimization^ (-) 
Safety^ (+) 
Teacher Morale^ (+) 
Planning and Action^ (+) 
Smooth Administration^ (+) 
Rule Clarity (+) 
Rule Fairness (+) 
Planning and Action (+) 
Safety (+) 

Pet. measures improved 



b 


3 


3 


1_ 

D 


0 


0 


b 


3 


3 


b 


■0 


0 


b 


2 . 


2 


b 


0 


0 


2 


3 


5 


1 


0 


1 


3 


3 


6 


0 


0 


0 


2 


3 


5 


0 


1 


1 


3 


2 


5 


0 


0 


0 


2 

Mm 


3 


5 


0 


0 


0 


4 


1 


5 


0 


0 


0 


4 


3 


7 


1 


1 


2 


4 


3 


7 


1 


0 


1 


4 


1 


5 


0 


0 


0 


2 


2 


4 


1 


0 


1 


3 


3 


6 


1 


1 


2 - - 


2 


2 


4 


1 


0 


1 


1 


2 


3 


1 


0 


1 


2 


3 


5 


1 


1 


2 


3 


3 


6 


1 


1 


2 


4 


3 


7 


1 


0 


1 


3 


2 


5 


0 


0 


0 


2 


3 


5 


0 


0 


0 


3 


2 


5 


0 


0 


0 


2 


1 


3 


0 


0 


0 


2 


1 


3 


0 


0 


0 


4 


2 


6 


1 


0 


1 


.69 


.79 


.74 


.50 


.20 


.34 



Note , The desired direction of change is indicated in parentheses after 
each scale name. All measures are taken from the SAES student surveys 
unless otherwise indicated. 

^ Measure taken from SAES teacher survey. 

^ Delinquency was not measured in 1981 for middle schools. 
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Table 5 



Rates of Suspension, Expulsion, Attendance and 
Withdrawal from School, 1979-1982 



Suspensions Expulsions Average Daily Withdrawals 

Attendance^ 

79-80 80-81 81-82 79-80 80-81 81-82 79-80 80-81 81-82 79-80 80-81 81-82 



C .rtenay M.S, 


.26 


.13 


.08 


.00 


.00 




.93 


.93 


.94 


.03 


.01 


.01 


Rhett M.S. 


.15 


.09 


.10 


.00 


.00 


.00 


.93 


.95 


.94 


.03 


.01 


.01 


Rivers M.S. 


.11 


.18 


.25 


.01 


.00 


.00 


.94 


.94 


.94 


.02 


.02 


.03 


Haut Gap M.S. 




.10 


.20 


.00 


.00 


.00 


.93 


.93 


.93 


.01 


.01' 


.01 


Laing M.S.''' 


.08 


.13 


.15 


.00 


.01 


.00 


.93 


.94 


.94 


.02 


.01 


.02 


Brown H.S. 


.^6 


.66 


.32 


.01 


.01 


.02 


.89 


.90 


.91 


.14 


.10 


.14 


Burke H.S. 


.09 


.18 


.14 


.01 


.01 


.02 


.90 


.91 


.92 


.11 


.10 


.10 


St. Johns H.S. 


M 


.90 


.87 


.02 


.02 


.02 


.93 


.91 


.92 


.10 


.09 


.10 


Charleston H.S. 


.24 


.34 


.48 


.01 


.01 


.00 


.92 


.93 


.94 


.11 


.05 


.07 



'Control School 

I 

'Average daily attendance of average daily membership. 
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Table 6 

* California Test of Basic Skills 
FATHE and Control School Scores, 1981 and 1982 



Z scoring in 

Scale Score bottom 

~ national 

1981 1982 quartile 

School M SD *M SD 1981 1982 



Reading 



Courtenay 


436.1 


80.8 


459 


. 5** 


82.7 


42.9 


32.9 


Rhett 


458.4 


92.7 


481 


.8** 


92.2 


31.0 


19.1 


Rivers 


446.2 


75.8 


466 


.6** 


71.6 


39.0 


31.7 


Haut Gap 


475.8 


77.9 


488 


.0* 


73.5 


34.1 


27.2 


Laing<a> 


514.4 


101.6 


536 


, 7** 


88.9 


21.7 


11.5 


Brown 


483.2 


86.3 


488 


.9 


87.7 


57.6 


57.1 • 


Burke 


509.5 


90.4 


523 


. 1** 


91.8 


50.2 


45.4 


St. Johns 


519.4 


94.3 


528 


.6 


98.8 


41.7 


38.7 


Charleston<a> 


567.0 


90.5 


575 


.7 


?3.2 


36.2 


30.0 


PATHE Schools 


484.3 


91.9 


495 


, 3** 


90.6 


43.8 


36.3 


Control Schools 


543".'7 


99.0 


554 


.9* 


92.9 


29.8 


19.8 


Langimge 


Courtenay 


452.9 


76.5 


470 


.4** 


81.1 


36.0 


25.4 


Rhetc 


471.7 


92.4 


489 


.1** 


85.6 


26.4 


18.4 


Rivers 


462.1 


76.4 


474 


.1* 


70.9 


29.5 


24.9 


Haut Gap 


480.1 


73.6 


490 


.0 


64.6 


29.3 


18.6 


Laing<a> 


513.6 


99.8 


527 


.6* 


81.0 


18.3 


U.7 


Brown 


501.0 


84.8 


504 


.7 


87.6 


43.6 


41.4 


Burke 


516.0 


90.1 


527 


.8** 


91.2 


37.3 


37.1 


Sc. Johns 


520.1 


89.2 


522 


.1 


93.3 


39.8 


39.3 


Charleston<a> 


573.9 


95.6 


578 


.7 


92.-1 


-22.4 


21.1" 


PATHE Schools 


494.3 


88.6 


501 


. 1** 


86.9 


36.0 


30.4 


Control Schools 


547.1 


101.9 


551 


.5 


90.0 


20.6 


16.1 


Math 


Courtenay 


437.5 


65.2 


451 


.6** 


70.2 


37.6 


28.8 


Rhetc 


448.1 


75.3 


465 


.5** 


71.4 


31.9 


23.8 


Rivers 


440.4 


56.9 


448 


.5* 


59.1 


39.6 


32.9 


HauC Gap 


450.2 


65.4 


463 


.5** 


60.9 


33.1 


26.6 


Laing<a> 


485.9 


88.8 


508 


• 7** 


82.8 


23.0 


13.4 


Brown 


473.8 


77.9 


477 


.2 


80.4 


53.6 


52.1 


Burke 


488.7 


81.7 


504 


.5** 


81.9 


47.6 


44.7 


Sc. Johns 


509.1 


85.8 


509 


.0 


94.9 


41.5 


37.1* 


CharlesCon<a> 


531.8 


85.3 


546 


.1* 


88.3 


36.1 


33.8 


PATHE Schools 


472.0 


80.0 


478 


.6** 


80.1 


42.0 


35.8 


Control Schools 


511.2 


89.8 


526 




87.3 


30.0 


22.5 


Total BaCCery 


Courtenay 


422.9 


78.9 


445 


.0** 


83.0 


46.3 


34.5 


Rhetc 


442.1 


95.5 


463 




89.1 


32.8 


23.4 


Rivers 


430.9 


72.9 


445 


.9** 


71.8 


44.8 


35.8 


Haut Gap 


453.1 


76.7 


466 


.1* 


69.0 


39.2 


29.6 


Laing<a> 


496.0 


107.0 


518 




93.6 


25.5 


13.0 


Brown 


477.7 


86.4 


482 


.1 


90.8 


60.4 


59.4 


Burke 


499.1 


91.3 


515 


.1** 


92.5 


53.6 


50.4 . 


Sc. Johns 


513.2 


94.0 


519 


.0 102.9 


46.3 


44.3 


CharlesCon<a> 


559.6 


92.1 


571 


.6 


96.5 


38.9 


33.6 


PATHE Schools 


472.7 


92.8 


481 


.9** 


92.3 


47.7 


40.3 


Control Schools 


530.8 


104.0 


543 


.3* 


98.5 


32.9 


22.4 



Note.: Table entries are based on all cases in the school 
each year for whom a valid test score was provided. N's 
range froa 277 Co 634 for 1981, and from 338 to 661 for 
1982. 

<a>Control School 

^Difference between 1981 and 1982 is significant aC the 
p<.05 level. 

**Difference between 1981 and 1982 is significant at the 
p<.01 level. 
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1 

? 1 

I 

t 

[ 



Rhett 
Target 
Control 

Rivers 
Target 
Control 



WHaut Cap 
N Target 
I Control 



C. A. Broun 
Target 
Control 

Burke 
Target 
Control 

St. Johns 
Target 
Control 

All Schools 
Target 
Control 



Table ? 

Means and Standard Dcvliitions tor PATllE 
Target and Control 
Groups on Measures of Goal I: Reduce Delinquency 



Number of Suspensions 
Officinl 



.11 .51 
.12 .« 



.37 .86 
.50 1.20 



.32 .78 
.17 .65 



.43 .81 
.35 .60 



.17 .47 
.08 .27 



.97 1.67 
1.26 2.19 



63 
51 



74 



62 
23 



75 
75 



01 
73 



76 
76 



.36 .92 523 
.40 1.15. 457 



.21 .41 43 
.37 ,49 30 



,38 .49 47 
.39 .49 41 



.17 .33 30 
.27 .46 15 



.33 .48 54 
.23 .43 56 



.25 .44 55 
.22 .42 46 



.52 .50 52 
.51 .50 51 



.33 .47 346 
.33 .47 299 



Number of 
Expulsions 





fl 


SD N 


M 


SD 


.N 


H 


SD 


Courtenay 
Target 
Control 


.12 
.09 


.36 82 
.28 80 


.35 
.30 


.48 
.46 


65 
60 


.01 
.00 


.11 
,00 



.01 .13 63 
.00 .00 51 



.00 .00 84 
.01 .12 74 



.00 62 
.00 23 



.00 .00 75 
,03 .16 75 



.01 .11 81 
.01 .11 78 



.01 .11 76 
.01 .11 76 



.01 .09 523 

.ni .in 4S7 



Self-reported Delinquency 

Total Serious _ "se 



Number of Disclnllnnrv Infr.ictions 



Hnjor 



Minor 



SD 



M SD N 



SD N H SD 



.15 
.11 



.14 
.13 



.13 
.19 



.09 
.15 



.13 
.14 



.19 
.14 



.12 
.11 



.14 
.14 



• ID 




. ij 


.18 


62 


.14 


.22 


62 


2.64 


1.40 


81 


5.62 


2.54 


81 


IS 


55 


10 


,18 


55 


.10 


.21 


55 


2;82 


1.48 


80 


5.39 


2.52 


80 


.21 


42 


.12 


.20 


42 


.16 


.26 


41 


2.28 


1.27 


61 


3.92 


1.51 


61 


,11 




• 11 


* LJ 


31 


.12 


.24 


31 


2.28 


1,14 


49 


4.26 


1.52 


49 


.13 


51 


.09* 


Ak 


51 


.15 


.18 


53 


2.62 


i;i4 


81 


4.27 


1,47 


81 






* IQ 




46 


'1 

t mi 


.27 


48 


2,61 


1.23 


72 


4.53 


1.66 


72 


















2.42 


1.15 


59 


4.51 


1.49 


59 


.12 


37 


.05 


.10 


37 


.l2 


.23 


40 


2.96 


1.40 


23 


4.13 


1.49 


23 


.17 


13 


.11 


,16 


13 


.18 


.22 


13 






























2.44 


1.22 


70 


3.99 


1.59 


70 


.15 


53 


".06 


.12 


53 


.26 


.28 


54 


. 2.26 


',97 


70 


4.41 


1.43 


70 


.17 


'55 


.09 


.15 


55 


.22 


.28 


55 






























2.25 


1.12 


75 


4.15 


1.53 


75 


.22 


59 


.12 


,21 


59 


.31 


'.33 


60 


2.44 


1.28 


72 


4,24 


1.73 


72 


.17 


54 


.08 


.16 


54 


.25 


.30 


53 














.13 


52 


.06 


.10 


52 


.25 


.26 


51 


2.26 


1.39 


69 


4.23 


2.00 


69 


.13 


47 


.06 


.12 


46 


.20 


.27 


47 


2.13 


1.05 


70 


4.27 


1.57 


73 


.17 


356 


.09 


.16 


356 


.20 


.27 


361 


2.43 


1.25 


496 


4.41 


1.87 


496 


.17 


300 


.10 


.17 


300 


.19 


.27 


302 


2.47 


1.24 


436 


4.53 


1.84 


436 



*Dlffercnce between target and control groups is significant at the p<.0^ level. 



'Mea.snrcs not available. 
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Tnble B 

Menus and Stntidnrd Deviations for I'ATIIF 
tirfiot atiJ Control 
CroupK on HtMSiirL'S of Coal II: Improvu Aitcndaiice 







- Spring '82 




X Days Tardy 
to Class - SprliiR 


No. of Attcnd.ince 
'82 related disc, infrar.. 


Nutiihur of 
iJlliidrimalji 
W SI) 'n 






Self- 


reported Nnn-.it 


endance 






Tot.i! 




I'lU'xniSL'd 


Cuttjnt; School 
M SI) N 


CuttlnR Clas' 


Total Index 

\ M SI) i: 


(1 


ill 


N 


M 


c>u 


[l 


VI 


Ml 


K 






M 


M SI) 1 














































Target 


.07 


.08 


81 


.06 


.06 


81 


M 


.08 


81 


1.54 


L48 


Bl 


.11 .31 


82 


..47 


.96 


75 


.49 1.04 7. 


) .51 


.67 




Control 


.07 


.09 


77 


.07 


.08 


77 


.05 


.10 


77 


1.3B 


1.29 


60 


.11 .J. 


'fin 




1 nil 


f L 


.36 .83 7 




.07 


72 














































Target 


.06 


.07 


S6 


.06 


.06 


56 


.01 


.04 


56 


.67 


.83 


61 


<22 .42 


63 


.37 


.85 


51 


.as 1.23 5 


i .65 


.80 


51 


Control 


.08 


.10 


^7 


.06 


.07 




.03 


.09 


47 


.82 


.90 


49 


.10 .J/ 




IB 


.95 


40 


.68 1.47 4 


.45 


.00 


40 
















































.09 


.U 


80 


.07 


.12 


80 




.06 


80 


.91 


1.04 


61 


.17 .37 


84 


.20 


.65 


64 


.40 1.07 6. 


, .27 


.60 




Control 


.13 


.IS 


70 


.10 


.16 


70 


.04 


.08 


70 


.80 


.14 


72 


.08 .27 


74 


.20 


.56 


55 


.25 .82 5. 


) .28 


.60 


54 


Hiiut flail 








































.49 




Target 


' .08 


.08 


59 


.03 


.04 


59 


.02 


.03 


59 


1.07 


.96 


59 


.08 .27 


62 


, .10 


.41 


51 


.24 .55 5 


) .26 




Contrnl 


.09 


.11 


22 


.OV 


.05 


22 


.01 


.02 


22 


1.13 


1.22 


23 


.00 .00 


23 


•.42 


1.12 


19 


.63 1.30 I' 


) .42 


.77 


1? 


i". A. arown 














a 




a 


























Tarpot 


.10 


.10 


70 


.09 


.10 


70 


a 


1.28 


1.26 


70 


.19 .39 


75 


.33 


.60 


61 


1.10 1.32 6 


.78 


.78 


60 


Control 


.11 




69 


.10 


.13 


69 


_ a 




a 


1.24 


.98 


70 


.25 .44 


75 


.52 


.86 


62 


1.19 1.28 6; 


.90 


.84 


62 


Burke 














n 


a 




























TarRct 


.09 


,10 


72 


.08 


.09 


72 


a 


1.48 


1.19 


75 


.20 .40 


81 


.39 


.58 


61 


1:27 1.05 6 


3 1.03 


.80 


6E 


Control 


.11 


.17 


69 


.10 


.15 


69 


n 


a 


a 


1.12 


1.17 


72 


.29 .45 


78 


.45 


.83 


62 


1.04 .99 5 


\ .93 


.79 


55 


St. Johns 










































56 




.09 


.10 


62 


.07 


.09 


62 


.0^) 


.1(1 




1.36 


1.35 


69 


.18 .39 


76 


' .30 


.78 


56 


.59 .95 51 


) .54 


.71 


Control 


.0^ 


.08 


61 


.07 


.07 


61 


.10 


.12 


61 


1.60 


l.<^2 


70 


.17 .38 


76 


.29 


.78 


55 


,87 1.39 5^ 


1 .56 


.77 


54 


All SchnnU 






































) .58 


.74 


415 


Tiirp.i't 


.08 


.10 


iRO 


.07 


.09 




.0'. 


.07 


^']^ 


1.20 


1.22 496 


.16 .^7 


523 


.32 


.71 


419 


.711.11 42( 


Contr"! 


.10 


.13 




.08 


.12 


M5 


.(!5 


.11) 


111 


I. IB 


LIS 416 


,17 


AS? 


.40 


.88 


359 


.72 1.18 35^ 


1 .59 


.7? 


356 



I 

u 

I 



,1 

Me.isures not av.ill.ible. 

'^Withdrawals are cnscs that left school before the close of the 81-82 school ye.ir for rtMsnn.s other th.m r.radiiaclnn fnm tliL» twelfth ^rade. 

iIoie--The h.ise N on which computations of official suspensions, expulsions and witlulriiwals are Uised is all cases randomly assigned to 

trratnont and control conditions minus 24 cases vim were not located In the schools as of October, 1981. The base N on which computations 

of nil other outcomes arc based is nil cases randomly iisslfined to treatment and control conditions minus 72 cases who had withdrawn from 

school .IS of February 1982 and for whoni no second semester data were provided. i ' 
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Table 9 

Means and Standard Deviations for 
Target and Control 
Croups on Measures of Coal IV: Decrease Academic Failure 



Self-reported 





English 




« 1(11(1 




Social Studies 


Science 




Average 






Crudes 




M 


SD 


N 


H 


SD 


N 


M 


SD 


N 


M 


SD 


N 


M 


SD 


N 


H 


SD 


N 


Courtenay 




























. JU 




2.37 




76 


Target 


.40 


.49 


81 


.36 


.48 


81 


.31 


.46 


□ 1 

81 


Jo 




81 


49 


81 


.80 


Control 




.49 


78 


.Jo 


/ Q 


10 


.26 


.44 


78 


.27 


.45 


7R 


M 


.50 


78 


2.62 


.97 


72 


Rhett 






























55 






52 


Target 


.2ii 


.43 


55 


.07 


.26 


55 


.36 


.48 


CP 

!>} 


.1/ 


TO 


53 


2.75**.74 


Control 


.38 


.49 


45 




• jo 




.49 


.50 


45 


.20 


.40 




.35 


.48 


46 


2.36 


.62 


41 


Rivers 


























.44 


.50 


73 


2.35 


.72 


55 


Target 


.37 


.49 


73 


.42 


.50 


73 


.46 


.50 


72 


.26 


.44 


73 


J Control 


.45 


.50 


65 


.42 


.50 




.44 


.50 


64 


.40 


.49 


65 


.54 


.50 


65 


2.27 


.85 


55 


' Haut Gap 
































2.56 


.75 


52 


Target 


.38 


.49 


55 


.11 


.31 


55 


.38 


.49 


55 


.18 


.39 


55 


.16 


.37 


52 


Control 


.« 


.51 


22 


.14 


.35 


22 


.41 


.50 


22 


.23 


.43 


22 


.27 


.46 


22 


2.52 


.90 


19 


C« A. Broun 




































61 


Target 


.52 


.50 


62 


.38 


.49 


48 


.12 


.33 


33 


.32* 


.47 


44 


.47 


.50 


64' 


2.13 


.50 


Control 


.67 


.48 


63 


.38 


.49 


55 


.13 


.34 


30 


.49 


.50 


45 


.58 


.50 


65 


2.21 


52 


62 


Burke 


































.66 


63 


Target 


.39 


.49 


64 


.31 


.47 


55 


.30 


.46 


53 


.53 


.51 


36 


.40 


.49 


65 


2.36 


Control 


.40 


.49 


60 


.30 


.46 


47 


.17 


.38 


47 


.30 


.46 


40 


.31 


.47 


64 


2.43 


.60 


56 


St« Johns 


































.64 


56 


Target 


.20 


.41 


59 


.35 


.48 


49 


.41 


.50 


56 


.43 


.50 


30 


.40 


.49 


60 


2.32 


Control 


.31 


.47 


58 


.48 


.50 


52 


.27 


.45 


55 


.37 


.49 


30 


.41 


.50 


Si 


2.34 


.67 


55 


All Schools 
































2.40 


.71 


425 


Target 


.36* 


.48 


449 


.29 


.46 


416 


.35 


.48 


405 


.31 


.46 


372 


.37 


.48 


453 " 


Control 


.« 


.50 


391 


.14 


,48 


365 


.31 


.46 


341 


.33 


.47 


325 


.43* 


.50 


401 


2.39 


.75 


360 



Ntimber of Academii 
Rel ated Disc. Infr ai 
H SD N 



jiwDIffcrence between target and control groups is significant nt the p^.Ol level. 

Hote-The base N on which computations of official suspensions, explusions and withdrawals nre ba.sed Is all cases 
randoialy assigned to treatment and control conditions minus 24 cases who were not located in the schools as o£ Octo- 
ber, 1981. The base N on which computations of all other outcomes are based is all cases randomly assigned to treat- 
ment and control conditions minus 72 cases who had withdrawn from school as of February, 1982 and for whom no second 
semester data were provided. 



.25 .54 81 
.28 .59 80 



.23 .50 61 
.14 .35 49 



.16 .37 81 
.18 .39 72 



.25 .54 59 
.13 .34 23 



.06 .23 70 
.06 .23 70 



.08 .27 75 
.11 .36 72 



.14 .39 69 
.11 .32 70 



.16 .42 496 
.15 .40 435 
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Table 9 (cont'd) 



lot 3 



Standard Scores on Spring '82 CTBS Test 





Reading 






Math 




Language 






Total 




M 




N 


M 


SD 


N 


M 


uU 


N 


M 


SD 




Courtenay 


























Target 


408.37 


73.68 


79 


405.32 


60.61 


78 


416.94 


78.87 


79 


387.45 


75.37 


78 


Control 


405.90 


60.85 


75 


406.25 


53.62 


V, 


415.36 


64.16 


76 


386.73 


62.10 


75 


Rhett 


























Target 


441.05 


70.36 


58 


436.83 


55.26 


58 


444.71 


69.07 


58 


421.83 


67.96 


'58 


Control 


422.11 


77.72 


47 


429.81 


50.90 


47 


430.36 


57.57 


47 


407.72 


62.22 


47 


Rivers 


























Target 


432.74 


66.06 


72 


421.74 


47.73 


72 


445.85 


58.25 


72 


411.51 


56.41 


72 


Control 


437.12 


69.52 


64 


423.89 


48.66 


64 


440.67 


57.64 


64 


412.80 


61.48 


64 


Haut Gap 


























Target 


437.36 


60.21 


55 


420.35 


59.18 


47 


437.67 


57.71 


55 


409.41 


59.51 


54 


Control 


410.19 


49.72 


21 


402.00 


49.26 


21 


432.14 


52.64 


21 


392.57 


46.47 


21 


C. A. Brown 


























Target 


474.26 


76.09 


62 


465.58 


76.82 


62 


493.63 


73.94 


62 


467.56 


78.32 


62 


Control 


475.34 


80.50 


64 


457.66 


65.71 


64 


479.47 


72.48 


64 


459.50 


74.97 


64 


Burke 


























Target 


473.66 


82.24 


65 


459.11 


75.56 


65 


466.94 


82.16 


65 


454.54 


79.75 


65 


Control 


482.21 


85.70 


58 


470.96 


62.06 


'57 
i 


478.86 


83.03 


58 


466.16 


74.82 


57 


St. Johns 


























Target 


492.10 


90.02 


50 


465.94 


94.52 


'54 


478. 76* 


82.14 


50 


470.34 


102.60 


47 


Control 


465.85 


81.74 


52 


453.98 


78.22 


i48 

1 


458.62 


76.06 


52 


441.57 


74.78 


47 


All Schools 












1 




t 










Target 


448.64 


78.72 


441 


437.21 


70.67 


436 


453.06 


76.06 


441 


429.04 


79.87 


436 


Control 


445.04 


79.44 


381 


436.53 


63.70 


377 


448.64 


71.97 


382 


425.50 


73.32 


375 



^Difference between target and control groups is significant at the p<.05 level. 
**DiffGrence between target and control groups is significant at the p<.01 level. 

Me— The base N on which computations of official suspensions,, explusions and withdrawals are based is all cases 
randomly assigned to treatment and control conditions minus 24 cases who were not located in the schools as of Octo- 
tfj ber, 1981. The base N on which computations of all other outcomes are based is all cases randomly assigned to treat- 
.gjM(^"and control conditions minus 12 cases who had withdrawn from school as of February, 1982 and for whom no second 258 
^mammji^x data were provided. | 
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I- Hi! rfVmvmviBmBfmmr* 



.17 .47 71 



UniEU«R« 




lotal 




H in N 


M 


SD 


N 


M .48 79 


.69 


.46 


78 


.70 .46 76 


.79 


.41 


75 



Self-reported 
Reading Ability 
M SD N 



1.40 .98 63 
1.45 .99 62 



.49 .49 $S 
.!> .» 47 



.43 .50 56 
.57 .50 47 



.48 .50 58 
.68 .47 47 



1.62 .85 
1.60 .97 



42 
38 



.#1 .il n M .49 7J .42 .50 72 .64 .48 72 1.35 .80 54 



,41 II .W .10 W .49 .50 55 .70 .46 54 1.38 .79 42 



M 


.49 


72 


.42 


.50 


72 


.64 


.48 


72 


.M 


.» 


64 


.S2 


.50 


64 


.64 


.48 


64 


.M 




W 


.49 


.50 


55 


.70 


.46 


54 




.u 


21 


.17 


.51 


21 


.86 


.36 


21 




.6) 


62 


.69 


.46 


62 


.77 


.42 


62 


.11 




64 


.76 


.43 


64 


.84 


.37 


64 


.« 


.19 


61 


.80 


.40 


65 


.88 


.33 


65 


.76 


.64 


$7 


.69 


.47 


58 


.86 


.35 


57 


.70 


.66 


47 


.66* 


'.48 


50 


.77 


.43 


47 


.77 


.62 


61 


4 W 


.34 


52 


.91 


.28 


47 



1.30 .95 47 



1.44 .70 18 



1.60 .75 57 



"•^ :5 S ;» W .7* W .84 .37 64 1.38 .84 52 

M .11 -W .« « -W " .W .ij M 1.50 .84 54 

M M H .W .44 17 .W .47 58 .86 .35 57 1.39 .70 49 

,n M W .W .46 47 .66** .48 50 .77* .43 47 1.33 .71 52 

,i Ti .77 .62 41 .W .34 52 .91 .28 47 1.41 .84 49 

f 

,41 6)6 .59«N».49 441 .70**.46 436 1.45 .83 364 
.46 177 .68 .47 38? .79 .41 375 1-^2 '88 315 

li.^**«w4 m tmtiimmdmml miMm mlim 2* mm* who wew not locotod In the achoo b as of OcLo- 
MU ICIm! ITSkiiTwSitl*- •t M oU-r wtcowf art bMo4 It all ca««i randoisly oMipnod to trea - 

^ I»-«IU«I«M. <«• «*ool of Fobruary, 1982 and for vho« no second 
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Table 10 

HOiiiis and Standard Deviations for PATIIK 
Tarj;et and Control 
CroMpn »n Otlu-r Survey Mc\if;ures 



Attachment to 



P.I rents 





M 


SD 


N 


H 


SD 


N 


H 


SD 


N 


Courtt'n.ty 




















Target 


.73 


.25 


62 


.70 


.21 


51 


.66 


.22 


76 


Control 


.78 


.19 


60 


.0/ 




51 


.66 




72 


Rhett 




















Taffiet 


.78 


.19 


37 


.75 


.15 


33 


.65 


.28 


52 


Control 


.77 


.22 


27 


.70 


.U 


27 


.68 


.19 


39 






















Target- 


.75 


.22 




.73 


.19 


38 


.62 


.23 


61 


Control 


.73 


.28 


39 


.70 


.19 


32 


.65 


.25 


54 


Haut Gap 




















Tar»!('t 


.79 


.22 


30 


.71 


.19 


27 


.64 


.27 


50 


Control 


.70 


.22 


12 


.67 


.22 


13 


.68 


.26 


17 


C> A. Broun 




















Target 


.83 


.18 


51 


.76 


.18 


45 


.66 


.24 


60 


Control 


.85 


.17 


52 


.80 




46 


.59 


.22 


59 


Burki* 




















Tar{;et 


.79 


.21 




.76 


.16 


41 


.60 


.22 


64 


Control 


.77 




^5 


.79 


.K 


39 


.62 


.?? 


56 


St» Mm 




















Target 


.80 


.21 


47 


.75 


.18 


41 


.59 


.25 


55 


Control 


.79 


.24 


50 


.73 


.U 


/5 


.66 


.23 


55 


All Schools 




















Target 


.78 


.21 


317 


.74 


.18 


276 


.63 




418 


Control 


.78 


.22 


285 


.73 


.17 


253 






v,2 



School 



H SD 



F.ducatlon.il Schuol Involvement In 

Expcctatiuiis Effort Hxt racurricnlar Acllv . 



Practical 



.76 .21 



M 
11 




Mi 








1 . // 


/J 


61 


2.57 


1.88 


70 


45 


2.92 


1.80 


52 


34 


3.41 


1.82 


39 


52 


2.76 


1.85 


62 


43 


2.76 


1.90 


55 


34 


2.56 


1.91 


52 


16 


2.59 


2.03 


17 


57 


2.08 


1.63 


6» 


58 


2.20 


1.38 


56 


59 


2.03 


1.50 


64 


48 

i 


2.70 


1.05 


54 


53 


2.41 


1.55 


54 


52 


2.07 


1.59 


54 



.74 .21 369 
.73 .23 112 



2.45 1.73 417 
2.58 1.76 145 



M 


oil 




M 






n oil 


u 
w 








.JO 


01 








.55 


.27 


67 


.36 


.23 


65 


1.40 .37 


57 














1 1^ nt 
l.jj .Jt) 


3/ 


.67 


.24 


34 


.24 


.25 


32 


1.40 .41 


23 


.61 


.29 


61 


.31 


.22 


56 


1.28 .42 


46 


.56 


.29 


48 


.24 


.20 


43 


i.31 .50 


37 


.58 


.29 


46 


.25 


.22 


42 


1.17 .47 


33 


.56 


.30 


14 


.28 


.21 


15 


1.25 .35 


14 


.65 


.26 


60 


.22 


.17 


56 


1.47 .^0 


54 


.60 


.29 


60 


.26 


.18 


58 


1.38 .48 


51 


.58 


.26 


63 


.21 


.17 


58 


1.40 .41 


43 




.28 


55 


.17 


.15 


55 


1.46 .36 


45 


.66 


.27 


52 


.21 


.18 


44 


1.41 .40 


47 


.69 


.26 


51 


.25 


.22 


49 


1.40 .39 


47 


.60 


.27 


394 


.27 


.21 


365 


1.34 .42 


320 


,61 


.28 


329 


.26 


.21 


317 


IIP .42 


274 



^Difference between tarpet and control Rroiips Is sjRnlf leant at t'ln "'/)5 levul. 



tjotc*-Thc b.isc N on whkli computations of official suspensions, expluslons and wlthtlr.walH arc baaed Is all cases 
randomly assigned to tr-Mtnent and control conditions minus 24 canes who were n'^t located In the schools as of Octo- 
ber, 1981. The base '1 on yhlcli computations of all other outcomes are based is all. cases randomly assigned to treat- 
ment and control conditions minus 72 cases who had 'Withdrawn from school as of Kebnury, 1982 and for whom no second 
semester data were provided. 

I 
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Table 10 (cont'd) 





Pnrentnl Emphasis 
on Education 
M SD N 


School 
Punlshncnts 
W SD N 


H 


School 
Rewards 
SD N 


Hnlc Fairness 


Perceptions of School Climatt! 

Rule Clarity St-Tch Interaction 


Dlsrcsiicct 
for Students 


Control of 
Environment 




SD 


N 


H 


SD 


N 


M SD 


N 


M SD 


N 


}\ 


SD N 


Court onay 














































.63 


.29 


68 


.40 


.25 


69 


.37 


.31 


69 


3.U 


1.41 


66 


3.67 


1.06 


66 


1.56 .95 


68 • 


2.83 1.39 


66 


.56 


.23 64 


Control 


.63 


.29 


62 


.45 


.30 


62 


.43 


.32 


61 


j.44 


1.22 


61 


3.67 


1.06 


57 


1.69 1.00 


64 


2.41 1.30 


59 


.62 


.20 60 


Rhott 












































■ Ttirpet 


.59 


.26 


39 


.33* 


.29 


45 


.36 


.33 


45 


3.40 


1.47 


42 


3.79 


1.14 


43 


1.61 l.OO 


46 


2.77 1.34 


43 


.56 


.24 37 


Control 


.(S6 


♦ 27 


32 


.49 


.33 


30 . 


.26 


.26 


31 


3.48 


1.50 


29 


4.00 


1.19 


28 


1.27 1.23 


33 


3.17 1.29 


30 


.62 


.20 28 


Rivi-rs 












































Lirget 


.64 


,30 


50 


.36 


.28 


53 


.36 


.34 


53 


3.36* 1.20 


52 


4.04* 1.09 


51 


1.51 I.IO 


57 


2.7M.28 


43 


.54 


.22 46 


Control 


.60 


.29 


42 


.37 


.22 


37 


.35 


.31 


38 


2.78 


1.53 


36 


3.57 


1.14 


37 


1.24 1.04 


41 


2.54 1.38 


35 


.57 


.18 38 


flaut Clip 












































Target 


.58 


.29 


33 


.24 


.25 


35 


.34 


.35 


36 


3.03 


1.54 


35 


3.69 


1.18 


32^" 


1.42 1.20 


38 


2.56 1.64 


J4 


.58 


.26 31 


Control 


.57 


.30 


15 


.32 


.27 


15 


.32 


.29 


15 


3.00 


1.57 


14 


3.60 


1.18 


15 


1.50 I.IO 


16 


3.00 1.51 


15 


.64 


.21 14 


C. A. Brown 












































Tarpot 


.50 


.33 


54 


.21 


.25 


56 


.30 


.30 


56 


3.29 


1.28 


51 


3.61 


1.06 


51 


1.53 1.24 


57 


2.25 1.15 


52 


.58 


.24 53 


Control 


.50 


.32 


55 


.20 


.26 


55 


.24 


.26 


56 


3.36 


1.24 


50 


3.47 


1.38 


53 


1.53 1.04 


56 


2.48 1.42 


52 


.50 


.23 53 


Bwrkc 












































Target 




.32 


49 


.24 


.26 


58 


.24 


.28 


59 


3.10 


1.38 


57 


3.85 


1.18 


53 


1.50 hl8 


59 


2.65 1.36 


55 


.57 


.22 43 


Control 




.29 


50 


.22 


.21 


47 




.30 


47 


3.51 


1.10 


43 


4.09 


1.09 


46 


1.67 1.15 


46 


2.26 1.42 


43 


.51 


.21 45 


St* Johns 












































Target 


M 


.26 


49 


.20 


♦28 


52 


.24 


.25 


52 


2.85 


1.38 


48 


^ 3.86 


1.11 


51 


l.fi3 1.07 


52 


2.83 1.42 


50 


.56 


.23 47 


Control 


.42 


.28 


51" 


.19 


.27 


53 


.27 


.32 


54 


2.fil 


1.50 


51 


3.54 


1.26 


52 


1.42 1.15. 


53 


2.88 1.50 


52 


,.54 


.22 ' 50 


All Schools 












































Tarpct 


.55 


/III 


]M 




.28 


3riB 


,31 


.31 


m 


3.13 


1.3H 351 


1."') 


l.li 




I.V, 1. 10. 377 


2.f.P 1.37 3'0 


.56 


.23 321' 


C«\iitn>l 


iji^ 




307 


'31',, 


.21) 


29') 


'Jl 


'31 


30? 


M') 


1.3:1 :'84 


3.-V1 


1.2(1 




1.:.0 1.10 309 


2.(,1 1.41 :86 


.56 


.21 288 



;int£"Thc base N on which conputatlons of official suspensions, cxplusioiis and withdrawals are based is all cases 
tnndoffily assigned to treatment and control conditions minus 24 cases who were not located in the schools as of Octo- 
ber, 1981. The base N on which computations of all other outcomes are based is all cases randomly assiRncd to tre.it- 
mcnt and control conditions minus 72 cases who had -withdrawn from school as of February, 1982 and for whom no second 
semester data were provided. 
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Table 10 (Cont'd) 







Belief in 




Negative 














Working 






Have a 


Rebellious: 






Rules 


Peer Influence 


Alienation 


Victimization 




for Pay 




job 






Autonomy 






SD 


N 


\\ 






M 












M 




U 


H 


SD 


N 


H 


SD ■ 


Courtenflv 










































Target 


.62 


.24 


61 


.27 


.23 


68 


.37 


.22 


62 


.25 


.22 


69 


.60 


.45 


70 


.36 


.66 


70 


.50** .30 5 


Control 


.57 


.25 


60 


.24 


.18 


66 


.43 


.27 


60 


.24 


.27 


60 


.58 


.50 


71 


.48 


.74 


70 


.68 


.28 6 


KlioLL 










































Tnrget 


.63 


.30 


39 


.22 


.21 


49 


.33 


.26 


38 


,19 


.25 


44 


.62 


.49 


52 


.46** .61 


. 50 


.57 


.35 J 


Control 


.59 


.19 


29 


.18 


.16 


36 


.44 


.20 


29 


.18 


.20 


31 


.50 


.51 


40 


.18 


.38 


40 


.67 


.31. I 


RivofiS 










































Target 


.55 


.26 


47 


.28 


.22 


58 


.40 


.20 


44 


.27 


.25 


51 


,61 


.49 


64 


.29 


.52 


65 


.57 


.27 4 


Control 


.60 


.22 


33 


.24 


.19 


46 


.38 


.22 


37 


.22 


.22 


37 


.47 


.50 


51 


.19 


.52 


52 


.60 


.29 3 


llaut Cap 










































Target 


.59 


.30 


30 


.21 


.22 


46 


.43 


.26 


29 


.16 


.21 


37 


.5! 


.50 


47 


.47 


.65 


49 


.6! 


.36 3 


Control 


.59 


.24 


13 


.24 


.24 


14 


.39 


.35 


13 


.27 


.28 


15 


.44 


.51 


16 


.39 


.78 


18 


.61 


.34 1 


\d* fit UlUWll 










































Target 


.62 


.27 


54 


.24 


.22 


, 59 


.26 


.20 


51 


.10 


.17 


54 


.37 


.49 


59 


.28 


.52 


60 


.59 


.35 5 


Control 


.67 


.24 


53 


.22 


.19 


59 


.27 


.23 


52 


.09 


.16 


56 


.37 


.49 


60 


.33 


.54 


60 


.56 


.30 4 


fturKC 














































.20 


43 


.21 


.20 


60 


.37 


.22 


45 


■ .10 


.18 


58 


.44* 


.50 


63 


.26 


.51 


64 


.64 


.28 4 


Control 


.68 


.24 


45' 


.21 


.19 


55 


.30 


.23 


43 


.10 


.16 


46 


.62 


.49 


58 


.46 


.60 


57 


.56 


.34 4 


St. Johns 










































Target 


.70 


.20 


49 


.17 


.14 


54 


,42 


.24 


43 


.11 


.14 


52 


.55 


.50 


56 


.39 


.68 


56 


.56 


.34 4 


Control 


.63 


.21 


50 • 


.22 


.20 


53 


.41 


.24 


50 


.13 


.22 


53 


.55 


.50 


54 


.28 


.60 


54 


.60 


.33 4 


All Schools 










































Target 


.62 


.26 


323 


.23 


.21 


394 




.r 


317 


.17 


.22 


365 


.53 


.50 


411 


.35 


.60 


414 


.57 


.32 30 


Control 


.62 


.23 


288 


.22 


49 


329 










.22 


298 


.51 


.50 


352 


,34 


.61 


351 


.61 


.31 " 



^Difference between target and control groups is significant .it the ;i<.05 level! 
**Diffcrence between target and control f^roups is signiriciint at lIio p .01, level i 



Notc--The base II on which computations of official suspend .is, cxplusuus and withdrawals are based is all cases 
randomly assigned to treatment and control conditions minus . . cascii who wore not located in the scliools as of Octo- 
ber, 198K The bnse N on which computations of all other outcomes arc b,'j:od is all caiie;; randomly assigned to treat- 
ment and control conditions minus 72 cases who had withdrawn from school . of February, 1982 and for whom no secon;: 
semester data were provided. ' 



Table 11 

Percent Students Receiving PATHE Services 
and Number of Contacts - Burke 
High School and other PATHE Schools 



Number of Contacts 

% rec'd 

Service Mean SD Media 



Burke High School 

All Target Students (N=86) 84.9 7.2 12.4 3.0 

Target Students who took survey (N=59) 94.9 8.6 14.1 3.4 

All Control Students (N=80) - 55.0 2.4 6.5 .7 

Control students who took survey (N=48) 62.5 2.& 7.9 .9 

Other PATHE Schools 

Ml Target Students (N=454) 90.7 6.6 , 6.4 4.9 

Target Students who took survey (N=310) 94.8 7.6 6.8 5.6 

All Control Students (N=384) 60.4 2.2 .2 1.0 

Control Students who took survey (N=264) 62.9 2.1 3.1 1.0 
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Tablo 12 



Fidelity Scale 

The rater cor.pares implecented activities and tasks as documented on the 
Iir.plriementation Report to the standards for implementation describe on 
this and subsequent pages. After checking off which of the standards have 
been met, a rating is assigned to each intervention using the following 
scale: 

4sexceeds standards 

3=meets standards completely or 
with approved adaptation ' • 

2=meets 50-99-tf of standards 

l=meets 1-49-j of standards 

O=does not meet standards/not implemented 



Intervention 



Standards 



Tutoring 



-outside tutors involved 
-regular' schedule of tutoring contacts 
by C.S. 

-target students included 



Counseling 



-RAP sessions plan on file 
-evidence of RAP sessions 
-evidence of at least 10 parent 
■ contacts 

-contact with at least 90% of 
students suspended . . _ 

-system established to monitor 
attendance problems 

-target students included in counseling 



Study Skills 



-plan on file 

-evidence of implementation 



Job Seeking Skills 



-distribution of materials 
-presentations to groups on job seeking 
skills or career awareness 



Services to Target 
Students 



Resource Room 



Faculty Inservices 



-individualized instructional plan/needs 
assessment or problem statement for 
every target student 

-letter sent to parents 

-positive parent contact for every target 
student 

-at least one contact with each target student 
-link' to summer enrichment program 



-needs assessment memo used after each 
grading period 

-resource room open house conducted 
-evidence of teacher use 



-participation in team building conference 
-teacher participation in Student Team Learning 

Learning training 
-inservice plan on file which addresses 

identified needs 
-evidence of implementation of plan 



•.School Pride 
Campaign 



-plan on file for wtole year 
-evidence of implementation 



Source - Birdseye, A. T. Evnlunclng the Implenentntlo n of n Dellngnoncv 
p;7vencton Program;: The PATHE Experience . Paper presented nt the 
Annual Meeting of the American Educational Research Associntlon, Hew 
York, March 1982. 



Intervention Standards 



Ciirriculun Review -CTBS analysis on file 

Campaign -plan to improve identified weakness on file 

-evidence of meetings with teachers re plans 

and weaknesses 
-failure data collected and used to identify 

students and teachers who need help 
-help provided identified teachers and students 
-mini-tests in use 

-instructional materials and teaching strategies 



distributed 
-assistance in use of curriculu:.: guides 
-at least one teacher has adopted S.T.L. in 

classroom 

-S.T.L. monitoring forms completed . 
-test-taking motivational activities implemented 




Field Trips 


-plan on file 

-one trip for each of three areas (career, 
academic, cultural) 
-scheduled throughout the year 
-plannejl trips implemented 




Reading Experierxe 
Program 


-plan for school-wide program on file 
-evidence of implenientation 

-motivational activities planned and implemented 




Exploratory Program 


-schedule on file 

-display of explorator>- products in school 
-schedule implemented 



Discipline Policy -schoo> and classroom rules developed by teachers 

Review and Revision and students and posted 

-referral procedures and discipline referral form 
approved by principal 

-referral procedure implemented so that includes 
S.C.S. 

-assistance given to teachers with classroom 
management problems (at least 10) 



Peer Counseling -counselors trailed 

-plan on file 

-counseling services provided on regular schedule 



High school : 
-club plans or charters on file 
-club mid-year activity reports on file 
-end-of-year reports received 

Middle school: 
-clubs begun last year continue 



Student Leadership -meetings held regularly 

Team -statisfactory team plan on file 

-evidence of plan implementation 
-central leadership conference attended 
-participated in planning spring conference 
-attended spring conference 



Student Concerns 
Support Team 


-regular meetings held 

-contracts on file 

-satisfactory team plan on file 

-evidence of plan implementation 

-final evaluation meeting conducted 

Middle school: 
•'team member agrees to serve as liason 
-members agree to serve without pay next year 


Curriculum Support 


-regular meetings held 


Team 


-contracts on file 




-satisfactory team plan on file 


(same as above) 


-evidence of plan implementation 



Extracurriculuar 
Activities 
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Tnble 13 
Intensitv Rating Scale 



The rater compares intensity of progran service*^ as documented on the Imple- 
mentation Report tp the standards given below and rates each intervention using 
the following scale: 

4=exceeds standard by more than 
3=meets standard + 5-e 
2=meets 50-95' of standard 
l^meets 5-49* of standard 
0=meets 4o or less of standard 



Intervention 



Intensity Standard 



Tutoring 



400 incidents of tutoring 



Counseling 



600 incidents, of counseling 



Study Skills 



100 incidents of participation 



Job Seeking Skills 



1 alj school activity or 50 incidents 
participation 



Services to target 
students 



8 contacts with every target student 



Resource Room and 
services to teachers 



# teachers checkouts=2 X 

# teachers 



Faculty Inservices 



3 school level, 5 teachers attend 
S.T. L., and 5 attend-team building 



School Pride 
Campaign 



Curriculum Review 



6 all school activities 



incidents C.S. faculty contact=4 X 
# teachers 



Field Trip Program 



3 field trip conducted 



Reading Experience 
Program 



entire school, monthly, for more than 
15 minutes 



Extracurricular 
Activities 



H.S. - 10 club plans 

M.S. - 100 incidents contact 



StUiieiit Leadership ' 
Team 



incidents attendance = # team members X 6 



Studerii Concerns 
••.Supporr. Team 



40 incidents attendance 



CuvriculuiD Support 
Team 



40 incidents attendance 



Career Exploration 
programs tT.T C.) 



120 incidents attendance 



Business Education 
Partnership 



6 meetings or documented, contacts, 
(assembly, planning session) 



Parent Leadership 
Team 



30 incidents attendance 



Exploratory Program • 


entire school, weekly 






Discipline Policy 


a discipline referral 


£orms=# 


suspensions 


Review and Revision 


+ !i # suspensions 






Peer Counseling 


100 incidents contact 


related 


to peer 




counseling 







S ource ; Birdseye, A. T. Evaluating the Implementation of a Delinquency 
Prevention Program! The PATHE Experience . Paper presented at the 
Annual Meeting of the American Educational Research Association, 
New York, March 1982. , , . ^^ j^^ 







Intervention 


Standards 




Career Exploration 


-students recruited to attend Trident 

-follow-up meeting with students held 
-students recruited for spring program 
-students recruited for FACET 




Business Education 
Partnership 


-team formed with school aiiO buslTiess members 

-PATriE specialists included on team 

-plan developed to improve school management 

-implementation of plan evident 

-plans for summer meetings on file 




Parent Leadership 
Team 


-regular meetings held 
. -satisfactory team plans on file 
-evidence of implementation of plan^ 
-leadership training attended 
-participation in -spring training 
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Draft Table U 

Avera(;e of Intensity and Fidelity Ratings for Each PATHE Intervention 



Courtenav 


Rhe 1 1 


Rivers 


Hnut Cap 


All 
Middle 


Brown 


Burke 


St. Johns 


All 
High 


All 
Schoo 


Team Average 


1.7 


2.7 


1.9 


2.2 


2.f 


2.4 


3.3 : 


2.9 


2.9 


2. 


Student Leadership 


3.0 


3.5 


3.5 


3.0 


3.2 


4.0 


4.0 


3.5 


3.8 


3. 


Student Concerns Support 


1.5 


3.5 


2.0 


2.5 


2.4 


2.0 


2.0 


: 3.5 


2.5 




Curriculum Support 


1.5 


3.5 


2 .0 






9 n 


TO 
J . U 


J . U 


2.7 




Business-Education Partnership 


0.0 


0.5 


0.0 


0.0 


0.1 


1.0 


4.0 


2.0 . 


2.3 


1 . 


Parent Leadership 


2.5 


2.5 


2.0 


3.0 


2.5 


3.0 


3.5 


2.5 


3.0 


2.- 


Affective Average 


2.8 


2.3 


2.6 


1.9 


2.4 


2.2 


3.2 


2.2 


2.6 


2. 


Counseling 


3.5 


2.5 


2.5 


1.5 


2.5 


1.5 


3.0 


2.5 


2.3 


; : 2 . 


School Pride Campaign 


3.5 


0.0 


3.5 


2.0 


2.2 


3.5- 


•3.5 


■2.5 


3.2 


2. 


Field Trips 


2.5 


2.5 


2.0 


3.0 


2.5 


3.5 


3,0 


3.0 


3.2 


2. 


Discipline Review & Revision 


3.0 


3.0 


3.0 


2.0 


2.8 


2.0 


3.0 


3.0 


2.7 


2 .; 


Peer Counseling 


2.0 


2.0 


0.5 


1.5 


1.5 


0.0 


3.5 


2.5 


2.0 


1. 


Extra-curricular Activities 


2.0 


^.0 


4.0 


1.5 ' 


2.9 


3.0 


3.5 


/ 0.0 


2.2 


2. 


Academic Average 


2.8 


3.0 


1.5 


2.8 


2.5 


2.7 


3.6 


2.7 


3.0 


: 2. 


Tutoring 


1.5 


3.0 


2.5 


0.5 


1.9 


2.5 


3.5 


2.0 


2.7 


' 2 . 


Study Skills 


3.5 


3.5 


1.5 


2.5 i 


2.P, 


2.5 


4.0 


2.0 


2.8 


2. 


Resource Room 


1.5 


2.5 


2.5 


3.0 


2.4 


2.0 


3.5 


2.5 


2.7 


*? 


Curriculum Review & Revision 


3.5 


4.0 


2.0 


4.0 


3.4 


4.0 


4.0 ■ 


3.5 


3.8 


3. 


Rending Experience Program 


3.5 


2.5 


0.5 


3.5 " 


2.5 


2.5 


3.0 


3.5 


3.0 


2. 


Exploratory 


3.0 


2.5 


0.0 


3.0 


2.1 


_a 


a 






2. 


Career Average 


2.0 


0.0 


0.0 


3.0 ' 


1.2 


2.8 


3.0 


3.0 


2.9 


2.1 


Job-Seeking Skills 


2.0 


0.0 


0.0 


3.0 1 


1.2 


3.0 


3.5 


3.0 


3.2 


2.1 


Career Exploration Programs 


a 


a 


a 




a 


2.5 


2.5 


3.0 


2.7 


2.; 


Other Average 


1.3 


1.0 


1.0 


2.0 ' 


1.4 


2.5 


2.5 


1.8 


2.2 




Services' to Target Students 


1-5 


1.0 


1.0 


1.5 1 


1.2 


1.5 


1.5 


1.5 


1.5 


1.' 


Faculty Inservices 


2.0 


1.0 


1.0 


2.5 , 


1.6 


3.5 


3.5 


*2.0 


3.0 


: ^ 2. 


Total Implementation Average 


2.4 


2.4 


1.8 


2.3 1 


2.2 


2.5 


3.2 


* 2.6 


2.8 


. . 2.; 



This intervention was not implemented at this level. 

! 
i 
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«• 
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2.U 


144 


1.43 
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? 


.09 
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.•1 


» 


.M 


t 


.01 


11 


••1 


«» 


0 




1 


.01 
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«4NI 


10 




2 


.02 


Ml 


«ti 


.42 


l» 




12 


.15 
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*ll 


10 


.^2 


12 


.IS 


• 


M 




• 


.00 


0 


.00 


H 


«ii 




Ift 


.19 


9 


.11 


i«l 


.» 


.It 


41 


.»?•* 


14 


.20 


ft 


.It 


.If 


II 


.17 


11 


.14 


• 




.M 


0 


.00 


0 


.00 


f 


.01 


.61 


1 


.01 


I 


.01 






.0* 


t 


.02 


1 


.04 


• 




.M 


0 


.00 


0 


.00 






.M 


• 


.00 


0 


.00 


# 


*m 


.(M 
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.0 


.00 
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.04 


11 


.I4«* 


1 


.01 










4.«« 


241 


3.35 
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Tnble 16 

PATIIE Specialist Contacts, 1981-82 
UheCt Middle School (N«»510) 







Entire School 


Target 


Students 


Control 










Avg. 




Avg. 




Avg. 








no. 




no. 




nO'. 






% 


contacts 




contacts 




contacts 


Purpose 


No. of 


students 


per 


No . of 


per 


No. of 


Der 


of contact i 


contacts 


contacted 


student 


contacts 


student contacts 


Student 


Total Affective 


1986 


• 65 


4 .04 


498 


7.90** 


130 


2.50 


Counsel ing 


480 


.32 


.98 


220 


3.49** 


37 


.71 


Dlsclpl inc 


218 


.29 


.44 


53 


.84 


28 ■ 


.54 


Student Leadership 


234 


.19 


.48 


25 


A 0 


Q 
0 


.15 


Team 
















Peer counsclee 


0 


A A 

.00 


A A 

.00 


1 


.02 , 


0 




Peer counselor 


58 


.03 


.12 


9 


.14 ■ 


0 




Field trip 


0 


.00 


.00 


0 


.00 


0 


.00 


Conference 


174 


.20 


.35 


98 


1.56** 


17 


.33 


Clubs /extracurricular 


179 


.06 


.36 


2 


.03 


0 




activity 
















Special program 


565 


.18 


.15 


72 


1.14 


32 


.62 


Rap session 


78 


.14 


.16 


18 


.28 


8 


.15 


Total Academic 


641 


.22 


1.30 


. 359 


5.69** 


17 


.33 


Specialist tutoring 


639 


.22 


1.30 


359 


5.70** 


17 


.33 


Peer tutee 


0 


.00 


.00 


0 


.00 


0 


.00 


Peer tutor 


0 


.00 


.00 


0 


.00 


0 


.00 


"Outside" tutoring 


0 


.00 


.00 


0 


.00 


0 


.00 


Study skills training 


0 


.00 


.00 


1 0 


.00 


0 


.00 


Test 


0 


.00 


.00 


0 


.00 


0 


.00 


Trident Technical 


0 


.00 


.00 


0 


.00 


0 


.00 


College 
















Central Staff 


0 


.00 


.00 


' 0 


.00 


0 


.00 


program 








1 






2.85 


lotal 


2637 


.68 


5.36 


861 


13.67** 


148 


^Difference between target and 


control students is 


significant 


at the p<.05 


level . 




♦♦Difference between target and 


control students is 


significant at the p<.01 


level. 
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Table 17 



PATHE Specialist Contacts, 1981-82 
Rivers Middle School (N=530) 







Entire School 


Target 


Students 


Control 


Students 








Avg. 




Avg. 




A\JCT 








no. 




no • 




liKJ • 


Purpose ' 




/<» 


contacts 




contacts 




con cac ls 


Nn nf 
ii U • O J. 


students 


per 


No • 01 


per 


HO . or 


per 


of contact 


UUllLcluLb 


con tac ted 


student 


contacts 


student 


^ #^ ^ o 

con cac L s 


Cf*11/10T1 t* 

b L uucn L 


Total Affer^^vp 


1 Z 1 o 


• 01 


Z • ZO 




H • / j^** 


1 AO 
1 o u 


9 1 3 


Coun se 1 ing 




• JO 


O A 

.80 


1 AA 
100 


1 • 7-3 ** ** 


77 


1.03 


Discipl ine 


108 


1 7 


. 


9 ^ 
z 3 


9Q 
• Z7 


Z J 


.33 


Student T.pnHpr«;Vii d 




• 1 o 


.JO 


1 O/i 

1 zH 


1 A A** 


<J z 


43 


Tpnm 

J. wdUl 


















1 

1 


• uu 


. UU 


A 

u 


• uu 


0 
u 


00 


Pppt" nniincpInT" 

^CC^ CULillDCXU^ 


0 


.00 


.00 


A 

u 


• uu 


0 
u 


00 


VipIH t*T"in 

fXCJ.U LLX^ 




.00 


.00 


A 
U 


nn 
• UU 


n 


00 


C* r\T\ roT"OTiPO 

v/Ulli. CL C11L>C 


AO 


.05 


.09 






n 
u 


00 

. \J\J 


Clubs /extracurricular 


176 


.06 


.33 


24 


\28 


5 


.07 


activity 
















Special program 


141 


.25 


.26. 


19 


.22 


21 


.28 


Rap session 


0 


.00 


.00 


0 


.00 


0 


.00 


Total Academic 


369 


.25 


.68 


169 


1.99** 


53 


.71 


Specialist tutoring 


.311 


.19 


.58 


162 


1.90** 


47 


.63 




0 


.00 


.00 


0 


.00 


0 


.00 


Peer tutor 


22 


.03 


.04 


0 


.00 


2 


.03 


"Outside" tutoring 


0 


.00 


.00 


0 


.00 


0 


.00 


Study skills training 


2 


.00 


.00 


0 


.00 


1 


.01 


Test 


0 


.00 


.00 


0 . 


.00 


0 


.00 


Trident Technical 


0 


.00 


.00 


0 


.00 


0 


.00 


College 
















Central Staff 


34 


.06 


.06 


7 


.08 


3 


.04 


program 
















Total 


1588 


.65 


2.95 


572 


6.73** 


213 


2.84 



^Difference between target and control students is significant at the p<.05 level. 
**Difference between target and control students is significant at the p<.01 level. 
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Table 18 



PATHE Specialist Contacts, 1981-82 
Haut Gap Middle School (N=446) 



Entire School 



Target Students 



Control Studei 









Avg. 




Avg. 










no. 




no. 








% 


contacts 




contacts 




Purpose 


No. of 


students 


per 


No. of 


per 


No. o: 


of contact contacts 


contacted 


student 


contacts 


student 


contac 


Total Affective 


510 


.36 


1.10 


237 


3.70** 


2 


Counseling 


112 


.14 


.24 


57 


.89** 


0 


Discipline 


70 


.08 


.15 


7 


.11 


1 


Student Leadership 


53 


.04 


.11 


5 


.08 


0 


Team 














Peer counselee 


4 


.01 


.01 


0 


.00 


0 


Peer counselor 


43 


.04 


.09 


21 


.33 


0 


Field trip 


33 


.07 


.07 


17 


.26** 


0 


Conference 


154 


.14 


.33 


120 


r.88** 


0 


Clubs /extracurricular 


0 


.00 


.00 


0 


.00 


0 


activity 














Special program 


37 


.08 


.08 


8 


.12 


1 


Rap session 


4 


.01 


.01 


2 


.03 


0 


Total Academic 


113 


.07 


.24 


7 


.11 


0 


Specialist tutoring 


86 


.02 


.19 


0 


.00 


0 


Peer tutee 


0 


.00 


.00 


1 0 


.00 


0 


Peer tutor 


0 


.00 


.00 


0 


.00 


0 


"Outside" tutoring 


2 


.00 


.00 


1 0 


.00 


0 


Study skills training 


23 


.05 


.05 


1 6 


.09 


. 0 


Test 


1 


.00 


.00 


' 1 


.02 


0 


Trident Technical 


1 


.00 


.00 


1 • 0 


.00 


0 


College 














Central Staff. 


0 


.00 


.00 


1 0 


.00 


0 


program 








] 






Total 


623 


.38 


1.35 


1 244 


3.81** 


2 



Av{ 
n( 

conti 

StU( 



^'Difference between target and control students is significant at the p<.05 level. 
. **Dif f erence between target and control students is significant at the p<.01 level. 
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Table 19 



PATHE Specialist Contacts, 1981-82 
Brown High School (N=659) 







Entire School 




Target 


Students 


Control 


Students 








Avg. 




Avg. 




Avi£ • 








no. 




no . 




no • 






% contacts 




contacts 




contacts 


Purpose 


No. of 


students 


per 


No. of 


per 


No. of 


per 


of contact contacts contacted 


student 


contacts 


Student 


contacts 


0 ^ U \A w & • ^ 


Total Affective 


760 


.38 


1.15 


197 


2.43** 


34 


.36 


Counseling 


210 


.18 


.32 


55 


.68** 


16 


.20 


Discipline 


27 


.04 


.04 


8 


.10 


2 


.02 


Student Leadership 


276 


.08 


.42 


33 


.41** 


2 


.02 


Team 
















Peer counselee 


0 


.00 


.00 


1 


.01 


0 


.00 


Peer counselor 


0 


.00 


.00 


0 


.00 


0 


.00 


Field trip 


103 


.15 


.16 


20 


.-.25* 


9 


.12 


Conference 


138 


.14 


.21 


77 


.95** 


5 


.06 


Clubs /extracurricular 


6 


.00 


.01 


3 


.04 


0 


.00 


activity 
















Special program 


0 


.00 


.00 


0 


.00 


0 


.00 


Rap session 


47 


.05 


.05 


2 


.02 


7 


.09 


Total Academic 


.424 


.20 


.64 


116 


1.43 


28 


.36 


Specialist tutoring 


257 


.10 


.39 


100 


1.23 


18 


.23 


Peer tutee 


29 


.02 


.04 


10 


.12 


0 


.00 


Peer tutor 


60 


.03 


.09 


0 


.00 


• 1 


.01 


"Outside" tutoring 


2 


.00 


.00 


2 


.02 


0 


.00 


Study skills training 


0 


.00 


.00 


0 


.00 


0 


.00 


Test 


4 


.00 


.01 


1 


.01 


0 


.00 


Trident Technical 


72 


.10 


.11 


3 


.04 


9 


.12 


College 
















Central Staff 


0 


.00 


.00 


0 


.00 


0 


.00 


program 
















Total 


1203 


.46 


1.82 


314 


3.88** 


62 


.79 



*Difference between target and control students is significant at the p<.05 level. 
**Difference between target and control students is significant at. the p<.01 level. 
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Table 20 



PATHE Specialist Contacts, 1981-82 
Burke High School (N=1005) 







Entire School 


Target 


Students 


Control 


Students 












Avg. 




Avg. 












no.. 




no. 






% 






contacts 




.".ontacts 


Purpose 


No. of 


c t- iiH pn t* *; 


per 


Kr • of 


per 


No . -I r 


per 


of contact 


contacts 




c t* 11 r1 on t* 


cor;:..' •f.s 


student 


cont --.c . •■■ 




Total Affective 


1106 


.49 


1.22 


22t 


2.64** 


ILiO 





Counseling 


165 


.14 


.18 


73 


.85** 




• If 


Discipline 


494 


.33 


.54 


59 


.69 


52 


.65 


Student Leadership- 


182 


.06 


.20 


16 


.19 


i:^ 


.16 


Team 
















Peer counselee 


68 


.03 


.07 


8 


.09 


5 


.06 


Peer counselor 


60 


.03 


.07 


14 


.16 


4 


.05 


Field trip 


28 


.03 


.03 


4 


- .05 


3 


.04 


Conference 


62 


.06 


.07 


51 


.59** 


2 


-.02 


Clubs /extracurricular 


0 


.00 


.00 


0 


.CO 


0 


.00 


activity 
















Special program 


0 


.00 


.00 ' 


0 


.00 


0 


.00 


Rap session 


47 


.05 


.05 


2 


.02 


7 


. .09 


Total Academic 


883 


.15 


.97 


391 


4.55* 


94 


1.18 


Specialist tutoring 


810 


.13 


.89 i 


360 


4.19* 


94 


1.18 


Peer tutee 


0 


.00 


.00 


0 


.00 


0 


.00 


Peer tutor 


3 


.00 


.00 i 


0 


.00 


0 


.00 


"Outside" tutoring 


54 


.02 


.06 1 


31 


.36* 


0 


.00 


Study skills training 


0 


.00 


.00 


0 


.00 


0 


.00 


Test 


0 


.00 


nn 1 


0 


.00 


0 


.00 


Trident Technical 


16 


.02 


.02 


0 


.00 


0 


.00 


College 






1 










Central Start 


0 


.00 


.00 1 


0 


.00 


0 


.00 


program 






1 










Total 


1996 


.53 


2.20 I 


62.i 


7 . 22** 


194 


7.. 42 



I 

♦Difference between target and control students is significant at the p<.05 level. 
♦♦Difference between target and control students is significant, at the p<.01 level. 
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Table 21 

PATHE Specialist Contacts, 1^^81-82 
St. Johns High School (N=808) 



Purpose 
of contact 





Entire School 


Target 


Students 


Control 


Students 






Avg. 




Avg . 




Avg. 






no. 




no . 




no. 




% 


contacts 




contac t s 




contact: 


Nn nf 


students 


per 


No. of 


Der 


No. of 


per 




contacted 


student 




student 


contacts 


studen: 


1660 


70 


1 99 


375 


4.*81** 


113 


1.49 


411 


.27 


.49 


47 


.60 


48 


.63 


69 


.06 


.08 


8 


.10 


7 


.09 


151 


.08 


.18 


53 


.68** 


3 


.04 


6 


.01 


.01 


1 


.01 


1 


.01 


81 


.02 


.10 


4 


.05 


0 


.00 


0 


.00 


.00 


0 


.00 


0 


.00 


290 


.12 


.35 


205 


2.63** 


0 


.00 


5 


.01 


.01 


0 


.00 


0 


;00 


59 


.02 


.07 


4 


.05 


11 


.14 


588 


.49 


.70 ' 


53 


.68 


43 


.56 


491 


.31 


.59 


105 


1..''5** 


29 


.38 


250 


.16 


.30 ' 


72 


.92* 


20 


.26 


4 


.00 


.00 i 


2 


.02 


0 


.00 


47 


.02 


.06 


11 


.14 


1 


.01 


0 


.00 


.00 ' 


0 


.00 


0 


.00 


33 


.04 


.04 1 


5 


.06 


2 


.03 


62 


.07 


.07 ' 


7 


.09 


2 


.03 


94 


.08 


.11 1 


8 


.10 


4 


.05 


1 


.00 


.00 ' 


0 


.00 


0 


.00 


2279 


.77 


2.73 ' 


495 


6.34** 


155 


2.04 



Total Affective 
Counseling 
Discipline 
Student Leadership 

Team 
Peer counselee 
Peer counselor 
Field trip 
Conference 

Clubs /extracurricular 

activity 
Special program 
Rap session 
Total Academic 

Specialist tutoring 
Peer tutee 
Peer tutor 
"Outside" tutoring 
Study skills training 
Test 

Trident Technical 

College 
Central Staff 

prograiL 

Total 



*Dif f eren 
**Dif ference 



ce between target and control students is significant at the p<.05 level, 
nee between target and control students is significant at the p<.Ul level. 
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virgin Islands Alternative Education Project 
J. St. John 



Abstract 

The Virgin Islands Alternative Education Project is one of a num- 
ber of demonstration pr oj ect s sponsored by the Office for Juvenile 
Justice and Del inquency Prevention as part of its Program in Del in- 
quency Prevention through Alternative Education. It is seeking to 
implement two interventions disseminated by the National Diffusion 
Network (NDN) — Focus and PATL. Interim evaluation resul ts suggest 
that Focus is being implemented largely as intended , but with some 
changes from the original Focus model. PATL is being implemented to 
a more limited degree, due largely to a limited number of teachers 
implementing the model. Despite some weaknesses in the evaluation 
design and measures, results also suggest that the modified Focus 
intervention resulted in students receiving higher grades than they 
otherwise may have received. Although project implementers believe 
they perceive positive results as a resul t of their activities, no 
other consequences of the Focus intervention were detected using 
measures obtained in the evaluation. This outcome may be a result 
of the small number of teachers implementing the intervention or to 
incomplete implementation. Some positive changes in the school cli- 
mate were observed, but there is no evidence in a comparison of stu- 
dents exposed in differing numbers of teachers trained in PATLind 
that the limited implementation of PATL has had detectable effects 
on students. Program development is continuing, as are efforts to 
strengthen the evaluation. 



Manor Program Design Changes 

The Virgin Islands Alternative 
Education project continued its 
efforts to implement the National 
Diffusion Network (NDN) Focus (Focus 
Dissemination Pro j ect , 1979 , 1980 ) 
and PATL (Pelan, Zirges, Scroggins, 
& Arterberry, 1975) programs during 
the 1981-82 school year. Major 
changes in the program included the 
opening of an Alternative Education 
building on the Elena Christian 
Junior High School campus. Although 
the building had only two of the 
four needed classrooms, the 



This report covers the period of 
September 1981 to June 1982. 



structural beginnings of a school- 
within-a-school were launched for 
the Focus Program. In addition, a 
teacher coordinator was appointed 
for the PATL Program and peer train- 
ing among teachers began anew. 

The staff could not attend the 
August, 1981 , planning workshop. 
Consequently their Program Develop- 
ment Evaluation (PDE) worksheets did 
not detail plans for the 1981-82 
academic year. Without these plans 
it was dif f icul t for us to know 
which program components they 
intended to implement or the degree 
of implementation success. For 
example , an intiensive publ ic rela- 
tions campaign was a component dur- 
ing the first implementation year. 
The campaign took on a new slant the 
second year because the school 
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system incorporated alternative edu- 
cation and there was no longer a 
need to lobby for survival. This 
development resulted in a change in 
project priorities. 

Maior Forcefield Changes 

In the fall, the director of the 
project , representing the Commis- 
sioner's Office , recommended the 
advisability of the services of a 
clinical psychologist on the proj- 
ect. He suggested that psychologi- 
cal testing (including intelligence 
testing) be used as a part of the 
selection process ■ and that indivi- 
dual behavior management programs be 
based on the use of psychological 
tests. The underlying philosophy of 
this approach is counter to the phi- 
losophy of project staff and was 
regarded as counter to the philoso- 
phy of Focus as well. In addition, 
the cost of implementing this recom- 
mendation was prohibitive.. There- 
fore, after negotiations with the 
ps . chologist , the project decided 
not to implement most of the recom- 
mendations, although changes in 
record-keeping procedures were 
adopted. 

The State Department of Education 
created a division of Alternative 
Education in the summer of 1981. 
During the school year the depart- 
ment recruited staff, chose c^rrricu- 
lum, and began program planning with 
several junior high schools in the 
system. The funded in-school coor- 
dinator helped the department estab- 
lish its new division and consulted 
with other schools. 

By late Spring, the in-school 
coordinator had been promoted to 
Director of Alternative Education 
for the Virgin Islands school sys- 
tem. One of the program's teache'i, - 
took on the project coordinator 
responsibil ities. The spring al so 
brought faculty requests for trans- 



fer to a n- V school expected to open 
in the fall. All but one of the 
Focus-tra: led staff requested trans- 
fer, and by Augu.^! the new in-school 
coordinator knew she would have at 
least three untrained teachers join- 
ing the staff. Fortunately, new 
teachers who were acquainted with 
Focus and were committed to its 
ideals joined the staff. New train- 
ing sessions were hel d with NDN 
trainers during the first weeks of 
the 1982-83 school year while stu- 
dent recruitment was in progress. 
Staff began the upcoming school year 
with excitement and optimism. 

Implementation 

We attempted to assist in program 
planning and PDE updating during 
1981-82 by phone and mail. This 
method was time consuming, and 
resulted in incomplete documentation 
of proj ect plans and their execu- 
tion. ' As " the * school" year" " pro- 
gressed, the Focus staff devoted 
considerable time to discussion of 
the Project Director's recommenda- 
tions (discussed earl ier) , and this 
may have slowed further development 
of the Focus intervention. 

Our obf rrvations suggest that 
more monitoring of the Focus inter- 
vention in the cl ssroom; more in- 
school cooi^.inator attention to 
attendance, discipline; and more 
attention to seeking family assist- 
ance with school discipline problems 
would have been useful. 

PATL Component 

PATL is a teacher training pack- 
age intended to a) help teachers 
learn alternative teaching and dis- 
cipl inary methods , and b) confront 
their attitudes toward students and 
the effects these attitudes have on 
students. The training program pro- 
ceeds in several st eps. First , 
teachers who volunteer to teach the 
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curx'iculum participate in an 
introductory four-day workshop. 
After the workshop, trainees form 
groups of about five members. PATL 
has four training "kits" — training 
group members work together to com- 
plete the kits. Each kit has learn- 
ing objectives that must be mastered 
sequentially. Mastery is demon- 
strated by a 90% proficiency rating 
on an in-classroom observation of 
the techniques. The PATL program 
has four stages of implementation 
that correspond with the completion 
of each kit. In addition, there are 
three level s of implementation for 
each kit: (1) training completed, 
(2) in process of implementation, 
and (3) implemented. PATL develop- 
ers expect the program to begin 
affecting those students having 
PATL-trained teachers in at least 
three of their classes; minimally, 
those teachers must be implementing 
PATL at the first of the four imple- 
mentation stages. 

Plans to strengthen the 1981-82 
PATL component were made during the 
previous year when project managers 
realized the component was not being 
given sufficient attention. Manag- 
ers planned for the appointment of a 
Teacher Coordinator. The coordina- 
tor was to oversee the recruitment 
of new teachers for peer training 
and structure the program. The NDN 
PATL trainer was scheduled to return 
for a refresher workshop in Septem- 
ber. Each of these steps were 
implemented. The Teacher Coordina- 
tor reported , however , that peer 
training was not going as expected 
because peer trainers and trainees 
had difficulty finding time to meet. 
This was largely due to the school^s 
double session school day resulting 
in rigid schedules for teachers. 
Consequently, by late spring many 
trainees had reportedly covered only 
one or two of the objectives in the 
first .training kit. During an 
August, 1982, planning workshop, 
project managers reported that not 



all teachers who had participated in 
the PATL training were implementing 
the PATL techniques due to insuffi- 
cient knowledge. Therefore, imple- 
mentation was probably at level 3 of 
stage 1 for some teachers. 

Focus Component 

In March, 1982, I visited tho 
project site to learn what had been 
implemented. The assistance of an 
NDN consultant on focus was sought 
to prepare to assess implementation 
of the major program components of 
Focus before the visit. The consul- 
tant supplied a checklist of 11 com- 
ponents that included ranges of 
acceptable implementation. The fol- 
lowing on-site observations are out- 
lined according to NDN's checklist. 

1. Family group meeting . Focus 
calls for students to engage in 
daily problem-solving sessions 
together with Focus teachers. Stu- 
dents in the project were grouped 
into four "family" units. One class 
period each day was set aside for 
the family groups to meet sepa- 
rately. The observed family group 
was participating in a values clari- 
f ication exercise and not observed 
resolving the problems of members in 
that particular session. Staff mem- 
bers interviewed reported that the 
family meetings were handled differ- 
ently by each staff member, but they 
did not fully describe what occurred 
in the meetings. 

2. Student leadership board . 
Focus calls for a student leadership 
board to help resolve problems not 
solved in family meetipgs. Project 
staff thought this component should 
not be implemented until the project 
had been operating for several years 
and effects had taken hold. 
Instead, students were invited to 
attend staff meetings when problems 
needed attention. This change is 
viewed by the State Facilitator as 
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an adaptation of the program model 
rather as an interim strategy 
required while other aspects of the 
project develop. Indeed, the faci- 
litator views the entire Focus model 
as open to adaptation rather than as 
a prescriptive model for interven- 
tion design. 

3. Mini school . The Focus model 
calls for a designated Focus area be 
established within the school, and 
for Focus staff to have their own 
offices. At the project site, a 
campus Al ternative Education buil d- 
ing had been opened but two of the 
four Focus classes were being held 
elsewhere in the school building. 
Division of the two Focus classrooms 
into four rooms was scheduled for 
completion during the summ.er of 
1982. 

4. Parental involvement . Focus 
calls for "some kind of positive 
support" on the part of parents. At 
the Virgin Islands project, parents 
of project participants must sign 
permission slips before their chil- 
dren 'can be considered for selec- 
tion. In addition, proj ect plans 
call for te'acbers to report to 
parents their students' progress, 
and to seek parents' help with dis- 
cipl ine problems. Records reviewed 
on site showed that at least some 
parents were contacted by some 
teachers , but no systematic report- 
ing occurred. 

5. Program contracts . Focus 
calls for students to set individual 
contractual goals. At least one 
proj ect staff person had contracted 
with students in her group. The 
contracts served as a means of moni- 
toring and measuring progress. 

6 . Student ownership of the pro - 
gram . Focus sets up the expectation 
that students "will demonstrate loy- 
alty to the program in some way." 
Proj ect managers reported that stu- 



dents seemed to be exhibiting 
loyalty and cited student statements 
of support. At the end of the 
school year, however, some students 
requested they not be 1 is ted as 
Fpcus students on the school promo- 
tion list. 

7* Use of reality feedback and 
confrontation . Focus calls for the 
consistent appl ication of real ity-o- 
riented feedback, and staff and stu- 
dent confrontation of inappropriate 
behavior. Such an approach seeks to 
insure that students receive spe- 
cific and immediate feedback on 
their behavior and are given an 
opportunity to change . Some of the 
teaching staff were using these 
techniques and others were not. In 
one Focus classroom a student 
entered late and was wearing a hat. 
The teacher confronted his lateness 
and the student gave the teacher a 
late pass. Wearing a hat was 
against school rules and the teacher 
asked him what the hat was doing on 
his head. He promptly removed the 
hat. In another Focus classroom the 
same behaviors by the same student 
went without comment. 

8. Scheduling student's day . 
Focus call s for at least four hours 
of instruction a day in the program. 
Focus-trained staff should teach 
Focus students a Focus-developed 
family curriculum, and three of the 
students' regular academic subjects. 
Each project family group was sched- 
uled in five Focus- taught courses 
and in regular school courses for 
the rest of the day. The Focus 
courses were occupational relations, 
family, English, math, and social 
science . The occupational r el at ions 
course- was designed to expose stu- 
dents to careers of all types, help 
students develop job skills, and 
help students gain everyday practi- 
cal skills. 
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9. Coinmunitv involvement . Focus 
recommends that junior high students 
be involved in one community service 
project each semester. Senior high 
students should either participate 
in jobs or community service proj- 
ects. Over the course of implemen- 
tation, five project students have 
had jobs. A public relations cam- 
paign implemented by proj ect staff 
v/as substituted for the recommended 
student community service proj ect s . 

10. Assessment . Focus recom- 
mends that Focus participants be 
pre- and post-tested in "three main 
academic areas". Project managers 
improved on this simple pre- post 
evaluation design by attempting to 
implement a more rigorous quasi-ex- 
perimental design. Tliey added to 
the assessment measurements relevant 
to a delinquency prevention demon- 
stration proj ect . Next year the 
project will be implementing a true 
experiment and re-adopting the pre- 
post approach as a method of moni- 
toring student progress. 

11. Admission policy . Focus 
recommends that current participants 
and staff decide which new appl i- 
cants should be admitted. The proj- 
ect staff substituted an admissions 
policy that both identifies the 
troubled students the program is 
intended to serve, and allows for 
the formation of participant and 
control groups necessary for an 
evaluation (see below for a more 
detailed description of the selec- 
tion process). 

Other observations . The Focus 
program does not prescribe curricu- 
lum or classroom methods beyond the 
use of reality feedback and confron- 
tation; consequently each teacher 
was using either the required Virgin 
Islands curriculum, or curriculum of 
their own design for the family 
classes. Teaching methods were up 
to the teaching staff. For 



Virgin Islands 

instance , the Engl ish teacher 
reported using a highly individual- 
ized approach that was structured by 
test results, while the math teacher 
reported using a model where stu- 
dents studied when they wanted. In 
the Engl ish classroom, the teacher 
and student developed learning 
obj ectives, and students worked on 
individualized materials. In the 
math classroom, the teacher reported 
that student3 were given a specific 
time period to master as signed 
niaterial s from the regular math 
book, ^ and were tested after each 
unit. In both classrooms the teach- 
ers gave students individual atten- 
tion. 

On the day I observed the 
classes, attenia nee was low. While 
observing one class v/ith about 10 
students present , students from the 
other class in the al ternative edu- 
cation building were milling about 
on the porch . In a later Focus 
class, only three of the students 
present in the class of 10 observed 
earlier were on time. One more of 
the students came to class later. 

Observation limited to a single 
day is not, of course, a dependable 
way of describing a school. The 
typical day may have looked either 
better or worse than the one 
observed. 

Effectiveness 



Focus Evaluation Design 

The national evaluation staff and 
project staff agreed to stu<^y Focus 
effects by using a regression dis- 
continuity design (Cook and Camp- 
bell, 1979). Tliis design was chosen 
because it al lowed service s to be 
given to students most in need of 
the program. The project staff 
developed a screening device on 
which referring teachers were asked 
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to rate students' academic 
abilities, and indicate attendance 
and behavior problems. Students' 
scores in the first two waves of 
eligible referrals were ranked, and 
the median score determined the 
selection pool. Tliose scoring above 
the median became Focus partici- 
pants , and al 1 others became the 
control group. Students referred to 
the program later had to score above 
the original median score to be 
selected as participants. This 
score was intended to serve as pre- 
participation information to be used 
in statistical analysis. 

Project staff executed each step 
for design implementation as 
planned, but the design broke down 
because of a v;eak relation between 
screening device scores and out- 
comes, (interpr etabl e regression 
discontinuity designs depend upon a 
strong relation between pre-treat- 
ment measures and outcomes.) 

Selection procedures for the 
regression discontinuity design had 
resulted in non-equivalent groups. 
Table 1 illustrates that small dif- 
ferences between Focus and compari- 
son students are apparent for grades 
in the first quarter of the school 
year (before the Focus interventions 
were begun in earnest) and a moder- 
ate difference in attendance . We 
had forseen the possibiliry of 
design failure and had administered 
a brief version of the jchool Action 
Effectiveness Study (SAES) student 
questionnaire in early January, 
about five school weeks after most 
students were admitted to the pro- 
gram. 

Administration of the SAES' ques- 
tionnaire pretest a coupled with' the 
controls originally identified for 
the regression discontinuity design 
created a quasi-experimental non-e- 
quivalent control group designr The 
pretest included SAES scales measur- 



ing Alienation, Bel ief , School 
Effort , Practical Knowledge , Posi- 
tive Self -Concept, School Attach- 
ment, Self-reported Total and Seri- 
ous Del inquency , and Sel f-repor ted 
Drug Involvement. Items on self-re- 
ported school and class skipping, 
and suspensions were also included. 
These seal es and items were used 
either because the proj ect was 
expected to affect these outcomes, 
or because they are known to be 
strong correlates of delinquency. 
Scores were derived as they had been 
for SAES scales in the Spring of 
1982 — summing the standardized 
scores of the items comprising the 

,^ scales (Gottf redson, Gottf redson, 

'Ogawa, and Rickert, 1982). 

School data from the first school 
quarter were also used as pre-parti- 
cipation measures because the proj- 
ect staff were recruiting and sche- 
duling students at that time, and 
formal "implementation" did not begin 
until the second quarter of the 
school year. Although the question- 
naire pre- test did not occur until 
J-^nuary, it was also used as a 
source of "control" variables in the 
analyses because of the non-equiva- 
lence of the treatment and compari- 
son groups. Its use. is a conserva- 
tive procedure . Any 
pre-parti cipation measures (either 
school or questionnaire-based) that 
correlated with both treatment and 
outcome were used as statistical 
control s for outcomes measured at 
the end of the year. 

Outcome Measures 

The SAES questionnaire was admin- 
istered in late April. The project 
evaluator reported school and class- 
room attendance , school grades at 
the end of quarters one and two, and 
again after the second semester was 
compl eted. School discipl ine data 
were reported at the end of first 
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and second quarters. Metropolitan 
Reading Test scores from the fall 
and spring administrations and 
police contact information for the 
previous year were reported in June 3 
1982. Unfortunately, 47% of the 
Spring achievement test data for the 
Focus students were missing, making 
these data of little value. 

Results 

Focus . Mean scores on the ques- 
tionnaire administered in Spring , 
1982 , and means for outcomes meas- 
ured through archival records are 
shown in Table 2. The grade point 
average (GPA) for Focus participants 
was significantly (p<.01) higher 
than the grade point average for the 
comparison students. No other dif- 
ference shovm in Table 2 is statis- 
tically significant. Two other out- 
comes measured by spring 
questionnaire items differed signi- 
ficantly. Self-rt>ported grades 
favored the Focus group (p<.05), and 
Focus students reported less class 
skipping than did the comparison 
students (p<.01). 

Recall that the comparison group 
was not equivalent to the Focus 
group at the outset, however. In 
other words, the treatment and com- 
parison group differed systemati- 
cally at the outset in ways that 
would lead them to differ at a later 
time in systematic ways regardl ess 
of whether or not they were exposed 
to any intervention. In such cir- 
cumstances it is necessary to use 
statistical techniques to adjust 
post-- intervention differences fcr 
differences to be expected on the 
basis of these pre-existing differ- 
ences. Table 3 shows the unadjusted 
association (r) of Focus participa- 
tion with GPA, self-reported grades, 
and sel f-reported class skipping. 
It al so shows the association when 
statistical control s are used to 
adjust for the measures of pre-ex- 



isting differences amojig students 
(beta) . When statistical control s 
are applied, only the association of 
Focus participation with GPA remains 
statistical ly significant . The 
adjusted associations observed for 
the other two outcomes, al though in 
the postive direction, are not sta- 
tistically significant. 

PATL . Table 4 describes PATL 
outcomes grouped according to the 
number of teachers exposed to PATL a 
student had, That is, 'if a student 
had one teacher who had completed a 
four-day workshop, he or she is 
listed under "One PATL Teacher." If 
^the student had two such teachers, 
he or she is included in the "Two 
PAll Teacher" group. The table 
classifies students in this way^ 
irrespective of the extent of con- 
tinued peer training and of degree 
of implementation of PATL methods by 
teachers. No differences are sta- 
tistically signficant. There is no 
consistent relation between the num- 
ber of teachers exposed to PATL and 
student outcomes. 

D iscussion 



Focus 

. New project managers were asked 
whether the improved grade point 
average of Focus participants was 
indicative of mastery of the subject 
matter, or reward for effort . They 
believed that the students' grades 
were based more on effort than mas- 
tery because the Focus philosophy 
calls for the use of such rewards as 
motivators for further improvement. 
Because of this, the reisults for GPA 
may represent a measure of implemen- 
tation, and may not address the 
question of masteiy. 

The most direct way to assess 
mastery is through the use of stan- 
dardized achievement tests. I3nf or- 
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tunately, standardized achievement 
test scores were available for only 
53% of the students, and could not 
be used to address the issue of mas- 
tery. The evidence is clear that 
grades where higher than they other- 
wise would have been for treatment 
students; the interpretation of this 
result is necessarily ambiguous. 

Observations of the impl ementa- 
tion of Focus are encouraging . 
Although not yet implemented as per 
NDN specifications, 1981-82 Focus 
implementation improved substan- 
tially over the previous year. For 
1982-83, the new in-school coordina- 
tor is committed to implementation 
improvements . She is especial ly 
committed to developing strategies 
for increasing class attendance. 
Finally, the managers' commitment to 
a stronger experimental design 
should reduce evaluation problems 
and give a better picture of 1982-83 
effects. 



PATL 

VATL implementation did not occur 
as planned. The project plans to 
strengthen both the PATL training 
component, and the implementation of 
PATL concepts after training is com- 



pleted. The new in-school coordina- 
tor plans to work closely with the 
PA 111 teacher coordinator to remove 
obstacles to peer training. Peri- 
odic visits to classrooms of trained 
teachers are expected to improve the 
the match between program design and 
implementation. 

School-Level Results 

Results reported elsewhere (Gott- 
fredson, Gottfredson, & Cook, 1983, 
Chapt er 5 ) show some * changes in 
school, cl imate between 1981 and 
1982 . Specifically, increases were 
observed in teacher job satisfac- 
^^tion, teacher non-authoritarian 
attitudes, and students' reports of 
school rewards. Decreases were 
observed in self-reported school 
grades, students' reports of commu- 
nity crime, and students' reports of 
drug involvement and del inquent 
behav ior more general ly * Al though 
the detailed ihterpretation"of' these 
results is difficult, they are 
encouraging. Decreases in reports 
of classroom disruption, which we 
expected to observe because project 
staff believed this to be a conse- 
quence of placing disruptive stu- 
dents in the Focus program, were not 
observed. 
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Table 1 



Means and Standard Deviations for Focus and Comparison Group 
First Quarter 1981-82 Pre-Intervention Measures 



Measure 



Focus 



M 



SD 



Comparison 



M 



SD 



Grade point average — 
all subjects 

Grade point average — 
core subjects 

Total school days absent 



.'1.59 .76 51 1.41 .72 49 

1.43 .80 51 1.19 .73 49 

7.10 8.16 41 3.68 6.03 37 
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Table 2 



Means and Standard Deviations for Focus and Comparison Group 

Spring 1982 



Focus Comparison 



Outcome measure 


M 


SD 


N 


M 


SD 


N 


Questionnaire-based 










• 




Alienation 


.47 


.24 


25 


.49 


.25 


28 


Attachment to parents 


.52 


.28 


30 


.57 


.28 


31 


Belief 


.57 


,, .28 


28 


.64 


.26 


28 


Interpersonal competency 


.68 


.27 


28 


.72 


.24 


26 


Negative peer influence 


.28 


.20 


30 


.21 


.18 


30 


Parental emphasis on education 


.52 


.30 


29 


.49 


.34 


28 


Positive self-concept 


.61 


.15 


24 


.56 


.20 


24 


Practical knowledge 


1.31 


.45 


28 


1.13 


.45 


26 


Rebellious autonomy 


.50 


.30 


26 


.53 


.28 


27 


School attachment 


.58 


? .22 


26 


.66 


.27 


31 


School effort 


.52 


.32 


29 


.61 


.27 


29 


School involvement 


.22 


.19 


29 


.25 


.17 


26 


School punishment 


.27 


• .33 


29 


.26 


.26 


26 


School rewards 


.39 


.32 


29 


.35 


.36 


26 


Self-reported delinquency — 














total 


.12 


.15 


25 


.10 


.12 


27 


Self-reported drug involvement 


.14 


.22 


26 


.12 


.15 


27 


self-reported serious 














delinquency 


.10 


.15 


25 


.07 


.13 


27 


Victimization 


.22 


.25^ 


29 


.20 


.26 


26 


School records 














Disciplinary infractions 


a 


a 


a 


a 


a 


a 


Grade point average 


1.44 


.92 


49 


.97 


.74 


45 


Total absence 


20.00 


22.78 


49 


15.55 


20.13 


44 


Police records 














Police contacts 


.09 


.29 


54 


.12 


.48 


51 



Data not reported 
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Table 3 



Correlation Coefficients and 
anu Standardized Regression Coefficients 
in the Regressions of Outcomes on Focus Participation 

Spring 1982 





Coefficients 




Measures 


r beta 


N 


School report of grades^ 


.267** .274* 


58 


Self-report of grades^ 


-.300* ~ .189 - ' 


-45 - 


Self-report of class skipping^ 


.324** .220 


46 



Note . Table shows the three outcome variables with 
significant correlations. N shown is the lowest 
pairwise n. 

a • 

First quarter school reports of grade point average 
and absenteeism, and January SAES measures of 
Alienation and Serious delinquency were used as 
statistical controls. 

First quarter school reports of absenteeism and 
class skipping, and January SAES measures of Alien- 
ation, Serious delinquency. School attachment, and 
Self-reported class skipping were used as statisti- 
cal controls . 

*p<.05 
**p<.01 
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Table 4 



Means and Standard Deviations for Students by 
Number of PATL-Trained Teachers, Spring 1982 



No PATL One PATL Two PATL , - Three PATL 

Teacher Teacher Teacltors Toachcrs 

Outcome measure M SD N M SD N M SD N M SD N 



Questionnaire-based 



Alienation 


.41 


.28 


18 


Attachment to parents 


.71 


.18 


20 


Belief 


.62 


.22 


19 


Interpersonal competency 


.83 


.17 


19 


Negative peer influence 


.15 


.17 


21 


Parental emphasis on education 


.63 


.29 


21 


Positive self-concept 


.72 


.20 


19 


Practical knowledge 


1.33 


.36 


19 


Rebellious autonomy 


.32 


.36 


19 


School attachment 


.76 


.23 


20 


School effort 


.68 


.26 


20 


School involvement 


.23 


.18 


18 


School punishment 


.17 


.26 


18 


School rewards 


.24 


.28 


18 


Self-reported delinquency — 








total 


.08 


.10 


18 


Self-reported drug involvement 


.09 


.10 


16 


self-reported serious 








delinquency 


.05 


.09 


18 


Victimization 


.09 


.14 


19 


School records 


a 




a 


Disciplinary infractions 




Grade point average 


1.76 


.94 


42 


Total absence 


8.52 


16. Z4 


40 


Police records 








Police contacts 


.02 


.15' 


45 



.40 


.22 


47 


.43 


.21 


44 


.42 


.27 


24 


.63 


.24 


50 


.64 


.29 


50 


.57 


.29 


25 


.63 


.26 


50 


.66 


.24 


44 


.62 


.25 


23 


.77 


.24 


49 


.77 


.28 


41 


.74 


.24 


24 


.17 


.18 


49 


.21 


.20 


50 


.20 


.15 


23 


.60 


.29 


50 


.61 


.31 


4^ 


.61 


.34 


24 


.68 


.21 


45 


.71 


.19 


39 


.67 


.17 


20 


1.24 


.44 


49 


1.14 


.41 


42 


1.19 


.44 


23 


.54 


.37 


49 


.58 


.33 


43 


.61 


.27 


22 


.66 


.23 


49 


.68 


.23 


49 


.65 


.21 


24 


.66 


.29 


48 


.64 


.26- 


49 


.62 


.29 


24 


.21 


.18 


45 


.23 


.15 


47 


.28 


.22 


20 


.20 


.22 


45 


.25 


.28 


46 


.25 


.29 


23 


.39 


.34 


45 


.25 


.27 


46 


.38 


.33 


23 


.07 


.11 


,46 


.07 


.10 


47 


.07 


.07 


18 


.09 


.14 


46 


.08 


.15 


48 


.10 


.14 


19 


.05 


.11 


.46 


.04 


.08 


47 


.02 


.04 


18 


.16 


.21 


45 


.14 


.18 


45 


.19 


.23 


22 


a 


a 


a 


a 


a 


a 


a 


a 


a 


1.49 


1.01 


1 93 


1.72 


.95 


89 


1.73 


.82 


46 


16.29 


23.15 


93 


9.01 


14.16 


88 


6.48 


10.35 


46 


.11 


.49 


100 


.02 


.14 


95 


.06 


.25 


47 



^ Data not reported 



Interim Quantitative Evaluation of the Academy for Community Education 

D. E. Rickert, Jr. 



The Academy for Community Educa-- 
tion is a small alternative school 
sponsored by the Office for Juvenile 
Justice and Delinquency Prevention 
as part of that Office's Program in 
Delinquency Prevention through 
Alternative Education. The Academy 
uses a token economy system, aca- 
demic education, prof essional/ voca- 
tional curriculum, and other inter- 
ventions described more fully by 
Daniels (1982, 1983) in providing 
services to youths at high risk of 
delinquent behavior drawn from the 
Dade County (Miami) Public Schools. 
The present brief report summarizes 
some interim quantitative results of 
the evaluation of this project as it 
operated during the 1981-82 school 
year. Qualitative results of the 
evaluation are described elsewhere 
(Daniels, 1983). 

Method 

Plans were made to assess the 
project using a non-random but 
closely matched control group for 
whom both pre- and post-intervention 
data were to be available.^ This 
design was partially implemented: 
Pre-and post-intervention data on 
absences, suspensions, tardiness, 
academic credits earned, standard- 
ized achievement test performance, 
and police contacts were available 
for analysis.^ The Academy expended 



1 . In its initial grant appl ica- 
tion, the Academy was to operate as 
a true experimental field trial , 
with eligible youths randomly 
assigned to treatment and control 
group conditions. For a number of 
reasons described by Daniels (1982), 
this expectation was not realized. 

2. No pre-treatment questionnaire 



considerable effort in creating a 
matched control group. Essentially 
the same screening procedures used 
to select students for participation 
in the Academy were used (at a later 
time) to screen students for the 
control group (Daniels, 1983). 

Analyses proceeded in several 
steps. First, treatment and control 
group .students were compared on the 
available pre-inteirvention data. 
And, because many students partici- 
,j pa t ed in th e Aca demy for onl y a 
short period of time (68.8%, or 75 
of 109, had at least six months' 
involvement in the Academy), compar- 
isons between long-term (six months 
or more) and control group students 
were also made on pre-intervention 
measure^. Second, comparisons of 
Academy per sisters and dropo^uts^were 
made to determine whether differen- 
tial attrition might bias the 
results. (Only 2 ':,2% of students 
ever enrolled in the project dropped 
out dur ing the year , a rel atively 
small percentage in projects of this 
kind.) Third, comparisons of treat- 
ment and control groups were made 
with and without the appl ication of 
covariates to increase the statisti- 
cal power of the analyses.-^ 



measures were available for the con- 
trol group, and post-intervention 
questionnaire measures were availa- 
ble for only 37% (21 of 57) of the 
control group students. An account 
of the difficulties in implementing 
the planned design may be found in 
Daniels (1983). The present report 
is limited to the most meaningful 
comparisons: those involving treat- 
ment and control group students on 
archival information. 
3. A covariate is a variable known 
to be correlated with the outcome 
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Results 

No statistidally significant dif- 
ferences were found between treat- 
ment and control groups on any of 
the pre-intervention measures avail- 
able for analysis. Nor were any 
statistically significant differ- 
ences found between long-term Aca- 
demy participants and the control 
group on any of these measures. 
These resul t s , summarized in Tabl e 
1, imply that the matching procedure 
implemented by the Academy resulted 
in a good match between treatment 
and control groups. Furthermore, 
such differences among the groups as 
are observed are relatively small in 
comparison to the individual differ- 
ences within groups (compare the 
differences between means in Table 1 
with the standard deviations which 
are indicators of the extent of 
individual differences) . 



variable of interest which is some- 
times introduced into an analysis to 
adjust for pre-existing differences 
between groups when the groups are 
not randomly equivalent. In the 
present case, as we shal 1 see, the 
treatment and matched control groups 
did not differ greatly on pre-inter- 
vention measures. Here the primary 
purpose in the use of covariates was 
to increase statistical power: The 
probability that a "true" difference 
between the groups will be detected 
by statistical analyses. The logic 
of the analysis of covariance, as 
this technique is called, is that 
some of the differences among indi- 
viduals in both the treatment and 
control groups can be accounted for 
by pre-existing individual differ- 
ences. By taking these individual 
differences into account , analyses 
can sometimes focus in more closely 
on group outcomes due to the treat- 



A comparison between students 
persisting in the Academy for at 
least six months and students who 
dropped out with fewer than six 
months'' participation is shown in 
Table 2. The table shows that per- 
sistence in the Academy is essen- 
tially unrelated to any of the pre- 
intervention measures examined. 

Post- intervention comparisons, 
between treatment and control group 
students are summarized in Table 3. 
Treatment students were suspended 
signiticantly fewer times than con- 
trol group students, withdrew invo- 
luntarily significantly more times, 
^^and earned significantly more aca- 
demic credits. Although not quite 
significant by conventional stan- 
dards , treatment students were al so 
tardy less often than were control 
students. No significant differ- 
ence s were found for absence s , num- 
ber of police contacts, or standard- 
ized achievement test scores. Table 
3 also shows that the same general 
pattern of results is present when 
only the long-term Academy partici- 
pants are included in the analyses, 
except that (of course) the exclu- 
sion of students who withdrew invo- 
luntarily from the Academy from the 
analyses alters the "result" for 
this form of withdrawal. 

A statistically more powerful 
form of analysis that takes pre-ex- 
isting individual differences into 
account is summarized in Tabl e 4 . 
In these analyses the results signi- 
ficantly favor the treatment group 
for number of absences, number of 
suspensions, number of times tardy , 
credits earned, and the Math Compu- 
tation and Reading sections of the 
Stanford Achievement test. The dif- 
ference in number of police con- 
tacts, although favoring the treat- 
ment group (especially for long-term 
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persisters), is not signifLcant. 
Even in these very powerful analy- 
ses, no significant differences were 
found for standardized test perfor- 
mance in the areas of math concepts 
or language. 

Discussion 

The present results are positive 
and impressive, but before elaborat- 
ing on their implications, some 
limitations must be described. The 
most important limitation is that, 
with the exception of police records 
and standardized test scores, these 
archival measures may involve some 
unknown degree of instrumentation 
bias. That is, it is possible that 
records of absences, suspensions, 
tardiness, and credits earned may be 
kept in somewhat different ways by 
the public schools on the one hand 
and the Academy on the other, or 
that the public schools and the Aca- 
demy may differ somewhat in the ways 
these outcomes are def ined. This 
limitation is, of course, largely 
unavoidable in a field trial such as 
this one. A second important limi- 
tation is that these analyses bear 
only on a part of the outcomes the 
evaluation seeks to examine. The 
unavailabl ity of questionnaire meas- 
ures is a serious shortcoming. 

The standardized achievement test 
data are especially important 
because, lacking questionnaire data 
collected in parallel ways for both 
treatment and control groups the 
only other data that were collected 
in perfectly parallel ways for both 
groups (pol ice contact s ) did not 
show significant differences. A 
significant difference favoring the 



treatment group was found for two of 
the four test scores examined. 

Despite these limitations, the 
following interpretations appear 
reasonable. 

1. Participation in the Academy 
results in less absenteeism than 
participation in the public schools. 
The difference, although not large, 
is significant. 

2. Participation irf the Academy 
results in fewer suspensions than 
would occur in the public schools. 
The difference is of moderate size. 

3. Participation in the Academy 
results in less tardiness than would 
occur if the students remained in 
the public schools. The difference 
is of moderate size. 

9 

4. Participation^ J.n_the^ Academy 
results in earning more academic 
credit than would be expected if 
students remained in the publ ic 
schools. The difference amounts to 
abcat 1.77 credits for students ever 
participating in the Academy, and 
amounts to about 2.74 credits for 
students who persist in the Academy 
for at least six months. 

5. During the 1981-82 school 
year, however. Academy participants 
withdrew involuntarily more often 
than students remaining in the pub- 
lic schools. 

6. Participation in the Academy 
results in higher scores on the Math 
Computation and Reading subtests of 
the Stanford Achievement Test. 
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Table 1 



Pre-Treatment Comparisons of ACE and Control Students 
1980-81 School Year 





Total ACE 




Lona- 


-term 


ACE 




Control 




Variable 


M 


SD 


N 


M 


SD 


N 


M 


SD 


1 


numoer or days aosent 


20.7 


19.8 




21 .U 


in T 

19 .7 


75 


19.9 


17 .7 


56 


Number of times suspended 


0.7 


1.2 


104 


0.7 


1.2 


75 


0.5 


1.4 


56 


Involuntary withdrawal 


0.0 


0.0 


103 


0.0 


0.0 


75 


0.0 


0.0 


5! 


Number of days tardy 


7.2 


7.4 


82 


6.7+ 


6.6 


59 


6.4 


6.1 


44 


Credits earned 


7.4 


3.4 


109 


7.4 


3.4 


75 


8.5 


3.4 


56 


Number of pol ice 


0.1 


0.4 


109 


0.0 


0.1 


""68 


0.1 


0.2 


51 


contacts 




















Stanford Achievement Test, 


national 


percen 


tile 














Math computation 


37.0 


22.5 


105 


36.9 


23.3 


72 


43.0 


24.9 


51 


Math concepts 


37.7 


22.9 


99 


38.1 


23.4 


70 


40.5 


28.8 


5( 


Language 


33.2 


25.0 


106 


30.1 


23.7 


73 


33.5 


23.4 


53 


Reading 


34.9 


23.3 


83 


33.0 


22.7 


64 


35.8 


26.4 


41 



Note , N'^s differ among outcomes due to missing data. Ws are as follows* disregarding 
missing data: ACE total, 109; ACE long-term, 75; control, 57. Long-term ACE participar 
had at least six months of treatment. 

+This mean is almost significantly different from the control group mean: £ < .06. 
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Table 2 



Pre-Treatment Comparisons of ACE Persisters and 
ACE Dropouts 



ACE Persisters ACE Dropouts 

Variable M SD N M SD N 



Kumber of days absent 


20.7 


18.9 


80 


21.3 


23.2 


26 


Number of times suspended 


0.8 


1.2 


79 


0.5 


1.0 


23 


Involuntary withdrawal 


0.0 


0.0 


75 


0.0 


0.0 


26 


Number of days tardy 


7.0 


7.0 


60 


8.2 


9.0 


20 


Credits earned 


7.6 


3.3 


80 


6.6 


3.5 


27 


Number of pol ice 


0.0 


0.2 


74 


0.3 


0.8 


22 


contacts 














Stanford Achievement Test, 


national 


percentile 








Math computation 


38.1 


21.7 


76 


'33T3 


~ 24.8 ~ 


~27 


Math concepts 


35.9 


21.9 


72 


44.2 


25.0 


25 


Language 


31.9 


23.5 


77 


36.0 


30.0 


27 


Reading 


34.9 


23.6 


70 


34.5 


22.6 


13 



Note, N's differ among outcomes due to missing data. N's are as fol- 
lows disregarding missing data: ACE persisters, 80; dropouts, 27. 
Long-term ACE participants had at least six months of treatment. 
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Table 4 



Adjusted Post-Treatment Means for ACE and Control Students with Pre-Treatment 
Data used as Covariates: 1981-82 School Year 



Total ACE . Long-term ACE 

Treatment Control Treatment Contro' 

▼triable M N M N M N M 



liaib«r of days absent 


16.82* 


108 


22.06 


52 


18.37 


75 


22.06 


Vwbttr of times suspended 


.16** 


103 


.65 


55 


.05** 


75 


.64 


Munbtr of days tardy 


7.86** 


75 


13.33 


31 


7.91** 


55 


13.32 


Cr«dita earned 


9.25** 


102 


7.48 


56 


10.18** 


73 


7.44 


Muib«r of police 


.17 


98 


.19 


56 


.03 


68 


.18 


contacts 
















Stanford Achievement Test, 


national percent 


:ile 




« 






Math computation 


42.07** 


96 


35.46 


44 


46.08** 


56 


34.47 


Math concepts 


40.42 


101 


40.46 


44 


44.00 • 


64 


40.24 


Language 


35.35 


111 


38.67 


44 


36.59 


69 


37.93 


leading 


38.90* 


83 


32.60 


41 


38.90* 


64 


31.70 



|fi££« N^s differ among outcomes due to missing data. N^s are as follows disregarding 
missiflK data: ACE total » 109; ACE long-term, 73; control, 57. Long-term ACE partici- 
pants bad at least six months of treatment. Analysis of covariance was performed usii 
tlie 1980-*81 variable corresponding to each 1981-82 outcome as a covariate. No expul- 
aiofis occurred for any control or treatment group student for whom pre-treatment data 
nert available. ' 

*Thla mean differs from the control group mean* at the ^ < ^05 level* 

I 

*^This mean differs from the control group mean at the ^ < .01 level* 

I 
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Alternative Education for Rural Indian Youth: Lac Courte Oreilles 



M. S. Cook 



Abstract 



The Alternative Education for Rural Indian Youth project (here- 
after referred to as the LCO project) was one of several projects 
funded by the Office of Juvenile Justice and Delinquency Prevention 
(OJJDP) as part of its Program in Delinquency Prevention through 
Alternative Education. The LCO project operated on the Lac Courte 
Oreilles Indian reservation, located in a remote part of . north-cen- 
tral Wisconsin, The project maintained two drop-in Youth Centers, 
installed PLATO computerized instruction, in the Youth Centers, 
developed a Youth Outreach effort, and partially supported PLATO 
instruction in the LCO reservation schools. 

The project sought to decrease delinquency on the reservation by 
providing recreation services to youth, and by decreasing drop-out 
from the LCO schools through the support of PLATO instruction, A 
variety of implementation difficulties including loss of funding for 
other reservation projects, staffing problems, unclear specification 
of goals and objectives, and a scattershot approach to program 
development resulted in the program not being fully implemented 
nearly two years after initial funding. The outcome evaluation is 
inconclusive due to a lack of data, but logical and theoretical 
analysis suggests that the project failed to implement a program 
that might reduce delinquency. 



Theory 



Letters of support for the grant 
application advocate for funds to 
maintain drop- in youth recreation 
centers (Youth Centers) to control 
del inquency , Al though the pr opo- 
sal 's goals and obj ectives and the 
proposed workplan do not mention the 
recreation centers, the letters of 
support suggest that the Youth Cen- 
ters were to be included as part of 
the .1 program, (More importantly, the 
Youth Centers evolved into the pri- 
mary project intervention). 



The LCO grant proposal was a mas- 
sive 500+ page work. Despite its 
length this volume contaius no cohe- 
rent statement of what the project 
intended to do , or why it intended 
to do it. An intent to implement 
PLATO, a proprietary computer-based 
instruction system, and to evaluate 
its effectiveness, was stated in the 
proposal ; the objective of imple- 
menting PLATO is puzzling, however, 
because the LCO school had had PLATO 
instruction available for three 
years prior to the submission of the 
grant proposal. 



Vagueness about what the program 
was about, and what it should do 
extended throughout the first proj- 
ect year. Week-long conferences in 
August 1980 and January 1981 were 
insufficient for evaluation staff to 
determine precisely what the project 
intended to implement. Over time. 



This report covers the period of 
September 1980 to June 1982, 
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project goals and objectives were 
developed, but their theoretical 
rationale remains sketchy. The fol- 
lowing account is constructed from 
Program Development Plans developed 
over the project ^s history, and from 
implications of the statements of 
project personnel. Bear in mind 
that this account is constructed 
post hoc , and was probably not used 
to guide day to day project decision 
making. 

According to project staff, 
delinquency on the part of Indian 
youth is due to having "insufficient 
opportunities for productive, rele- 
vant, and enjoyable activities" (LCO 
program development plan, dated 
March 16, 1982). There are "few 
opportunities for adolescents to 
engage in productive activity during 
weekend and evening hours" (same 
source ) • 

Although alcoholism and drug 
abuse were frequently mentioned as 
causes of delinquency by project and 
LCO personnel, the project only 
obliquely attempted to address these 
problems (see the discussion of the 
Group Home, below). 

Goals 

The goals of the project were to 
reduce delinquency among reservation 
youth, and reduce drop-outs from the 
LCO schools. 

Objectives 

Objectives of the LCO project 
included reducing the suspension 
rate among LCO high school students, 
reducing over the life of the proj- 
ect the number of students referred 
to the project from the reservation 
school, decreasing in-school absen- 
teeism among project-referred youth, 
increasing the number of youth par- 
ticipating in Youth Center activi- 
ties, and increasing the self-esteem 
of tribal youth. 



EKLC 



The foundation for strategies to 
meet these objectives is the belief 
that youth on the reservation get in 
trouble when they have nothing to do 
but "hang out . " Del inquency should 
be reduced if the youth have oppor- 
tunities available for constructive 
(or least not destructive) activi- 
ties : school or supervised recrea- 
tion. Youth in school or at a Youth 
Center should be protected from 
delinquency. 

The LCO schools (secondary and 
elementary) were founded in 1976 as 
an alternative to the Sawyer County 
public schools which, it was felt, 
did not provide an education in 
keeping with the Ojibwa Indian cul- 
ture. 

Early on, PLATO instruction was 
emphasized in an attempt to provide 
individualized instruction in a me- 
dium (visual and motor) that was 
believed to mesh with Indian youths^ 
behavioral and cultural style. The 
project proposed to expand the use 
of the PLATO system, and evaluate 
its effectiveness. PLATO^s theoret- 
ical import for del inquency preven- 
tion is that increased PLATO availa- 
bility will result in decreased 
abseenteism and drop-out from the 
school, which would produce less 
del inquency . (The appropriateness 
of computerized instruction is 
called into question because 
(a) prior to the start of the proj- 
ect the LCO high school already had 
eight terminals which were not in 
constant use , and (b) the principal 
of the LCO high school stated that 
absenteeism and drop out from the 
school were not really a problem.) 

The project contributed to PLATO 
instruction on the the reservation 
by placing four PLATO terminals in 
the Youth Centers, and temporarily 
putting four terminals in the LCO 
school s to supplement the eight 
they already possessed. The four 
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project terminals in the schools 
were ultimately withdrawn and placed 
in the Youth Centers. Because PLATO 
instruction was already a signifi- 
cant part of the LCO schools' cur- 
riculum, and the PLATO terminals in 
the Youth Centers were almost exclu- 
sively used for computer games (see 
below) , the importance of PLATO 
instruction as a part of the project 
is unclear. 

The actual primary project inter- 
vention was the staffing and mainte- 
nance of two recreational Youth Cen- 
ters. The objective here was to 
provide a place where youth on the 
reservation could engage in healthy 
recreation. The explicit rationale, 
was that the reservation was iso- 
lated and that youth on the reserva- 
tion needed structured recreation to 
protect them from delinquency (see 
Carlton, 1982, for a more detailed 
analysis of the social and geograph- 
ical environment of the LCO reserva- 
tion) . 

Is This Sensible ? 

Delinquency is a tough problem. 
It has not proven amenable to quick 
fixes or to ad-hoc programming. 
Evaluations of delinquency preven- 
tion projects have consistently 
noted that projects have not been 
powerful enough to solve the problem 
that they were intended to address 
(Wright and Dixon, 1977, Lundman & 
Scarpitti, 1978). The program as 
designed and implemented by LCO does 
not appear to us to be a credible 
approach to the prevention of delin- 
quency. Tlie primary theoretical 
principle behind the project is the 
maxim, "Idle hands are the devil '^s 
workshop." Acting on such a theo- 
retical premise , one intervenes by 
providing work or play for the idle 
hands : Youth Centers and computer 
terminal s . Yet, in an exhaustive 
review of del inquency prevention 
efforts, Dixon and Wright (1974) 



conclude that "...there is no 
evidence that these (recreational) 
programs in any way alter delin- 
quency" (p. 37) . Similarly, the, 
background paper for the Delinquency 
Prevention Through Alternative Edu- 
cation initiative (OJJDP, 1980) does 
not include recreational programming 
as among possibly effective 
approaches to the prevention of 
juvenile del inquency. More to the 
point perhaps, there is scant reason 
why recreational programs ought to 
affect del inquency . • Del inquency , 
and cr ime in general , take s 1 it tl e 
t ime . Vandal ism is usual ly an 
impulsive act taking only seconds. 
.^Shoplifting requires at most several 
minutes. Unless one wishes recrea- 
tion personnel to supervise youth 
every minute of every day (and call- 
ing such a program recreational 
would be strange), there is no rea- 
son to believe that youth will not 
have time avail able_ to ^commit crime 
if they so wish — no matter how much 
basketball or pool they might play. 

Obstacles to Program Development 

The LCO project suffered from a 
succession of dif f icul ties. From 
funding to staffing to politics, the 
proj ect confronted a continuous 
series of obstacles in the path of 
program devel opment • The fol lowing 
list of events, although not exhaus- 
tive of the trials that the LCO 
project went through, illustrates 
the project's struggles. 

Project Unfocused 

The proj ect ' s fundamental impl e- 
mentation problem was that its goals 
and objectives, and the strategies 
to achieve them, were diffuse. As 
noted above, the original grant pro- 
posal did not clearly outline a pro- 
gram. Neither the original project 
director, nor the director that 
shortly took over were involved in 
the proposal writing or project 
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design. They were not sure what 
they were supposed to be doing. 

After meeting with project per- 
sonnel in August 1980, and January 
1981, it was apparent to the evalua- 
tion staff that the project had no 
clearly specified aims. The project 
hoped to help youth on the reserva- 
tion stay out of trouble — anything 
that seemed to address that goal was 
fair game for project attention. 
However, after these conferences, we 
had no clear picture of the proj- 
ect's implemented or intended inter- 
ventions. 

Because the grant application 
stressed the importance of PLATO, 
and contained the stated objective 
of evaluating its usefulness with 
Indian children, a decision was made 
to evaluate PLATO instruction in the 
high school. Although PLATO had 
been an integral part of the LCO 
curriculum for some time before the 
project began, it was decided to 
administer an abbreviated form of 
the SAES national questionnaire to 
all students at the LCO high school 
once at the beginning of the school 
year and again at the end of each 
academic quarter. This plan would 
have allowed the project managers to 
monitor the usage of the PLATO ter- 
minals over time, and would have 
made possible a relatively weak qua- 
si-experiment of the effect of 
PLATO. This quasi-experimental 

design was not realized because of 
poor response on the early pretest. 
Early evaluation plans also called 
for monitoring the use of the youth 
centers in order to learn about what 
kinds of youths used the centers at 
which times. 

At an Augu s t , 19 82 workshop , 
project staff, along with evaluation 
and technical assistance staff , 
designed a new intervention: Youth 
Outreach (see below for a descrip- 
tion). An outreach effort seemed to 



encompass the myriad of activities 
that the project was involved in. 
Youth Outreach became a major focus 
of program development efforts. 
Unfortunately, going from paper to 
program takes time — interventions 
must be specified, agreements must 
be made, and staff must be hired and 
trained. For the LCO project, it 
was a matter of too little too late. 
As of April, 1982, when refunding 
decisions for the third project year 
were being made, LCO. was still hir- 
ing new proj ect staf f-»-more than a 
year and a half after the initial 
grant award. 

^ The unfocused nature of the ori- 
ginal proj ect proposal was exacer- 
bated by the governing tribal coun- 
cil's view of the project. They may 
have seen it as another aspect of 
general reservation activities and 
perceived little uniqueness to the 
del inquency prevention proj ect as a 
project. Thus funds were spent on a 
range of things such as repairing 
the school van, providing transpor- 
tation for school and Group Home 
events, staffing the Youth Centers, 
hiring alcoholism counselors, hiring 
outreach workers, taking youth on 
field and activity trips, renting 
PLATO connect time, buying PLATO 
terminals, and developing PLATO 
Ojibwa language software. While a 
subset of these activities might 
have been reasonable program 
efforts, the shotgun approach 
assured that no single intervention 
received enough attention to have a 
wel 1 defined impact . 

Loss of Other Federal Fundjjig 

A serious obstacle to the project 
was the large cuts that the reserva- 
tion was receiving in other federal 
funding concurrent with the start of 
the del inquency prevention proj ect . 
Instead of being able to focus on 
additional services to youth on the 
reservation, the project fell back 
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to maintaining some of the services 
no longer funded under other grants. 

As the project began CETA funds 
ran out . CETA workers had been 
staffing the two reservation Youth 
Centers. Concurrent with these 
cuts, the Youth Employment Training 
program, and the Youth Community 
Conservation Improvement program 
were discontinued. These federal 
projects had also provided some 
funding to maintain the Youth Cen- 
ters and their activities. At the 
state level, the Wisconsin Gover- 
nor's Council on Juvenile Delin- 
quency Prevention, which , had pro- 
vided the funding to build the Youth 
Centers in the first place , al so 
withdrew support funding. As a 
result of these funding cuts, the 
Youth Centers were cl osed on eve- 
nings and weekends — the times that 
the project felt were most crucial 
for delinquency prevention. There- 
fore, the project, instead of 
extending the services of the Youth 
Centers as they had hoped, fell back 
to simply keeping them open. 

Toward the end of the project, 
several other youth and education 
programs were being discontinued . 
As cuts in federal programs contin- 
ued throughout the life of the proj- 
ect, project staff became demoral- 
ized. They found themselves 
subj ected to requests to fund a 
variety of youth related events, 
whether they were linked to pr oj ect 
objectives or not. 

The situation deteriorated toward 
the end of the project. Along with 
the loss of the delinquency preven- 
tion project, the reservation lost 
funding for a fine arts project in 
the school, a youth athletic intra- 
mural s project, and an adult educa- 
tion project. 



Cuts in educational and social 
programs appear to have elicited a 
backlash on the part of the LCO 
reservation population toward whites 
that worked on the reservation. A 
tribal election in the winter of 
1982 changed the composition of the 
tribal council. The new members had 
campaigned on an "LCO jobs for LCO 
Indians" platform. In April of 
1982, 11 of the 17 white teachers in 
the LCO school system were fired, 
(with no replacements identified for 
the following year) al'ong with the 
principals of the elementary and 
high schools, the school superinten- 
dent, and the director of education 
, (who was a Native American, but was 
perceived as having let whites 
become too involved in the education 
of reservation youth). The alterna- 
tive education project director, who 
was white, was convinced that the 
only reason he was not fired was 
that the tribal ^council laiew^ that 
his dismissal would be the~ last 
straw as far as possible third year 
funding for the project was con- 
cerned. The project director 
believed that if the project had 
been refunded for the third year, he 
would have been replaced shortly 
after the grant was signed. 

Although the project director did 
not give up on the project during 
these events, it is fair to say that 
he was less invested in proj ect 
efforts past April 1982. The poli- 
tical shake-up particularly hurt the 
evaluation of the project, since the 
reservation firings were concurrent 
with attempts to survey users of the 
Youth Centers , and survey Indian 
youth who attended the local publ ic 
schools. Lacking the concerted 
attention of the project director, 
neither of these surveys met with 
much success (see below). 
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Succession of Proiect Directors 

As late as December 1980, four 
months into the project, no 
accounting procedures had been 
instituted and the paper work for 
the release of funds had not been 
completed. The project could not 
pay its bills, and was threatened 
with being dropped from the initia- 
tive. The project director's duties 
were assumed by the evaluation coor- 
dinator in December, 1980, after he 
had been associated with the project 
for only one month. The initial 
project director resigned in January 
of 1981, and the evaluation coordi- 
nator became the acting project 
director. The change in status of 
the evaluation coordinator produced 
three management difficulties: 
(a) there was now no one with the 
primary duty of overseeing the eval- 
uation, (b) having just been hired, 
the new director knew even less 
about the project's goals than the 
original ''director, and (c) the new 
director was white, which may have 
1 imited his abil ity to negotiate 
with the tribal governing board. 

Proiect Not Viewed as Distinct From 
Regular School 

Largely due to the unfocused 
nature of the proj ect , there was no 
clear distinction made between the 
al ternative education proj ect and 
the LCO schools. It may be that the 
original intention of the grant 
writers was that the LCO schools 
themselves were the "alternative" 
education part of the project. This 
is certainly plausible given that 
the LCO schools were indeed estab- 
lished as a cul tur al ly-or iented 
al ternative to the publ ic school s . 
Yet , the proj ect as devel oped and 
understood by evaluation and OJJDP 
staff was an entity distinct from 
the school. The confusion (whatever 
the source) resulted in the project 
having to justify expenditures that 



did not appear to OJJDP or the 
evaluation staff to be directly 
related to the interventions that 
the project was trying to implement. 

. An example of this situation was 
the development of the Ojibwa lan- 
guage program for use on the PLATO 
system. The argument advanced for 
underwriting the development of this 
computer software was that learning 
Oj ibwa would put the reservation 
child more in touch with his or her 
heritage, result in more self-es- 
teem, .and lead to less delinquency. 
This causal chain is plausible il a 
project is making a full-scale 
effort to improve the children's 
self-esteem. But that was not the 
project's intention. The point is 
that instead of concentrating on one 
task or another, the project delved 
into a variety of things whose only 
interconnection was that they had 
something to do with_ Indian_yquth^ 

Not Expecting Rigorous Evaluation 

A further organizational problem 
was that LCO grant administrators 
were familiar with a certain kind of 
federal evaluation. The LCO tribal 
organization was accustomed to fed- 
eral evaluators making site visits, 
and judging projects on the basis of 
interviews and audits. The evalua- 
tion of the alternative education 
project was perplexing. After 
decades of federal ly funded proj- 
ects, the project director commented 
that the evaluation of the alterna- 
tive educational project was the 
most rigorous that LCO had ever been 
subj ected to. 

The consequence of the reserva- 
tion's past history with evaluation 
was that evalution tasks were "just 
more paperwork. " When problems 
developed, or research designs broke 
down, the response was to assume 
that it didn't make any difference 
anyway. One result: Evaluations of 
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the Youth Centers and of the LCO 
high school school fell apart due to 
nonattention to data collection 
detail s. 

Change in Evaluation Staff 

A final obstacle to program suc- 
cess was the change in evaluation 
staff that the project had to 
endure. Project personnel were dis- 
satisfied with their original field 
worker. They were puzzled when he 
showed up for a site visit without a 
clear set of objectives for the 
visit . This event reinforced the 
project's belief that evaluations 
were generally meaningless . This 
attitude made evaluation efforts 
subsequent to the replacement of 
that field worker in the fall of 
1981 more difficult. Then, the 
replacement field worker resigned in 
January, 1982. In the two years of 
their project, therefore, they had 
three evaluation field workers. 
This lack of continuity, coupled 
with the project's previous experi- 
ence with federal evaluations, 
served to undercut attempts at for- 
mal, data-based evaluation. 

Summarv of Obstacles to Implementa - 
tion 

The major obstacles to the imple- 
mentation of the LCO Alternative 
Education Project were (a) the proj- 
ect had unfocused goals and objec- 
tives and expended funds on a grab- 
bag of interventions, (b) other 
federally funded projects were being 
discontinued, and the del inquency 
prevention project picked up some of 
their expenses, (c) project adminis- 
tration was al lowed to si ip in the 
early months of the project, (d) the 
proj ect was not viewed as an entity 
dist inct from the LCO school s , 

(e) project personnel did not expect 
a rigorous evaluation, and had no 
experience in conducting one, and 

(f ) there was a lack of continuity 



in the evaluation staff assigned to 
the project. 

The result of all these obstacles 
was that at the close of the LCO 
project it was still in an early 
stage of development. Staff was 
still being hired in March 1982, and 
all project components were not 
implemented until May of that year. 
By this time the decision had been 
made not to renew the project's 
grant award. 

Interventions 
Youth Center 

The fundamental intervention of 
the project was the staffing and 
maintenance of the reservation Youth 
Centers. There are t\70 Youth Cen- 
ters on the LCO reservation, located 
about 15 miles apart in the two most 
populous communities on the reserva- 
tion. The project initially hoped 
to expand the services offered at 
the Youth Centers , but f ol lowing 
cuts in other sources of funds to 
the Centers, had to be content with 
installing PLATO and paying the 
salaries of the Youth Center staff. 

The Youth Centers are drop in 
facilities that have grills, pool 
tables , video games , and meeting 
rooms. Space can be cleared to hold 
dances or other large gatherings. 
The proj ect hired persons to open 
and close the center, maintain the 
physical plant , and serve as cooks 
(see descriptions of the Support 
Specialists and Youth. Outreach Work- 
ers, below) • Project personnel also 
supervised the use of PLATO at the 
Youth Centers. 

The first direct project inter- 
vention at the Youth Centers (other 
than keeping the doors open) was the 
installation of on-line PLATO termi- 
nals. This intervention was 
intended to provide another source 
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of recreation at the Youth Centers 
(on-line PLATO terminals allow one 
to access a variety of computer 
games), and to make PLATO educa- 
tional instruction available for 
youth not involved in the regular 
LCO school program. 

Installation of the terminals got 
off to a slow start. The PLATO ter- 
minals had to be connected to a 
"parent" computer in Minneapolis, 
and the phone lines necessary to 
carry the information are different 
from the regular long-distance 
lines. New lines were installed, 
but this took four months. the 
lines were connected directly to the 
Reserve Youth Center, and regular 
long-distance lines carried the sig- 
nal to the New Post Youth Center. 

The necessity of sending the sig- 
nal by long-distance telephone lines 
from New Post to Reserve caused the 
project quickly to remove terminals 
from New Post. Long-distance rates 
were too expensive to justify the 
terminals' use at New Post. There- 
after, until the end of the project, 
the New Post Youth center was not a 
prime focus of project activities. 
Project funds were used to staff it 
and keep it open, but PLATO and 
other proj ect activities were una- 
vailable there. 

Because rental of connect time 
with the Minneapolis computer was 
expensive, the project eventually 
purchased self-contained PLATO 
micro-computers. Four of these were 
installed in the Reserve Youth Cen- 
ter in January of 1982. Although 
they were less expensive to run, the 
self-contained units had one major 
drawback; little software was 
available. 

The primary use of the PLATO 
units in the Youth Center was game- 
playing. They were seldom used for 
instruction. But game packages were 



not available for the self-contained 
units; The only software was 
designed for the teaching of basic 
academic skills — items of little 
interest to the Youth Center clien- 
tele. Project personnel estimated 
that use of the terminals dropped 
80% when the sel f-contained compu- 
ters were installed. 

Proiect-SDonsored Youth Commit - 
tees were developed in the summer of 
1981 . These committees were organ- 
ized by project personnel and were 
open to al 1 youth on the reserva- 
tion. They operated out of the 
Youth Centers. Participation in the 
.^meetings themselves was light, but 
the committees — or the project per- 
sonnel running them — did design and 
carry out a number of recreational 
activities for youth. From January 
1981 until April 1982, a total of 14 
events and trips were sponsored by 
the project^ JThese included dances, 
skating parties, field trips, skiing 
expeditions, and canoe trips. 

Youth Outreach 

In August 1981 a new Youth Out- 
reach intervention was designed that 
appeared to encompass a variety of 
project interventions. The LCO 
project decided to locate those 
youth on the reservation who were 
unemployed and not enrolled in 
school, or who were enrolled in 
school but in danger of being 
expelled for discipline or atten- 
dance reasons. The Youth Outreach 
effort was designed to identify such 
youth, offer counseling and tutor- 
ing, and involve these youth in con- 
structive activities. Two catego- 
ries of workers were responsible for 
this effort: Support Specialists, 
and Youth Outreach Workers. 

The first Support Special ist 
began work in November of 1980, and 
a second in April of 1981. They 
were originally responsible for 



-422- 



308 



LCO 



staffing and supervising the youth 
centers, and the PLATO terminals in 
them, and for providing drug and 
alcohol abuse counseling to Youth 
Center clients. With the advent of 
the Youth Outreach effort, they 
acquired the additional responsibil- 
ities for making home visits to Out- 
reach youth, and of supervising the 
Youth Outreach Workers. 

It is difficult to say how quali- 
fied the Support Specialists were to 
carry out their duties. Tlieir sta- 
tus as tribal members was certainly 
an asset; it would be difficult for 
outsiders to understand the economic 
and social problems that the reser- 
vation youth faced. On the other 
hand, although drug and alcohol 
abuse counseling was one of the Sup- 
port Specialists' primary responsi- 
bilities (indeed they viewed it as 
their most important repsonsibil- 
ity), neither Specialist had any 
training in counseling. Although 
academic training neither guarantees 
nor prevents one from attaini^tig the 
skills necessary to carry out effec- 
tive substance abuse counseling, a 
lack of training in counseling is 
not indicative of a strong counsel- 
ing component. 

The Youth Outreach Worker posi- 
tion was created to establ ish a 
liaison between the project and res- 
ervation youth. The Youth Outreach 
Workers were to be the people in the 
f ield identifying reservation youth 
in need of service, and organizing 
and supervising recreational activi- 
ties. The Youth Outreach Workers 
also staffed the Youth Centers, and 
were responsibl e for unspecified 
"peer counseling." 

The Youth Outreach Workers were 
all high school graduates or were 
currently attending the LCO high 
school. Youth Outreach Workers ran 
the Youth Committee meetings, and 
oversaw their activities. The Youth 



Outreach Workers were also 
responsible for conducting a census 
of Youth Center users, administering 
an evaluation survey to Youth Center 
cl ients , and carrying out a f iel d 
survey of reservation youth who 
attended the Sawyer County public 
schools. 

By January 1982, three Youth Out- 
reach Workers had been hired, and 
four more were hired in March 1982. 

Alternative Education ' 

Prior to the start of the alter- 
native education project, students 
^at the LCO high school who exhibited 
chronic absenteeism were expelled 
from the school. The project estab- 
lished a policy whereby students who 
attained seven un excused absenses 
were suspended from the school, and 
required to attend PLATO tutoring at 
the Youth Centers. If these stu- 
dents performed satisfactorily they 
could be reinstated in the regular 
LCO curriculum at the beginning of 
the next academic quarter. This 
referral and PLATO, tutoring compo- 
nent was called the "Alternative 
Education Program." 

Other Interventions 

Proiect funds contributed toward 
the PLATO instruction offered at the 
LCO high school and elementary 
school . Project funds paid for part 
of the rental of the PLATO terminals 
in the LCO high school. In addi- 
tion, in January 1982, the project 
purchased eight PLATO self-contained 
microcomputers. Two of these were 
placed in the LCO high school, and 
two in the LCO elementary school. 
In April of 1982, all of these 
self-contained PLATO computers were 
installed at the Reserve Youth Cen- 
ter. 

The project also supported the 
development of an Oj ibwa language 
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program for use on the PLATO system. 
The package describes in Ojibwa sev- 
eral traditional Ojibwa events (e.g. 
"The Sugar Camp"), while on the 
screen the scene is drawn and 
labeled in Ojibwa. Headphones allow 
the student to listen to the speech. 
Students respond by repeating the 
sentence they have heard (but with 
no feedback as to correctness), and 
engage in multiple-choice tests of 
matching written Oj ibwa to drawings 
of objects named in the previous 
lesson. A fundamental difficulty 
with the program is that Ojibwa 
grammar is substantially different 
from that of English — the language 
of all children on the reservation. 
For example, in Ojibwa, the verb is 
always the last word in the sen- 
tence. Unfortunately, the program 
does not address grammar at all. 
Because they are not instructed to 
the contrary, children make the rea- 
sonable assumption that the the 
first word spoken in Oj ibwa corres- 
ponds to to this first word of the 
English translations.' It doesn't. 
Observations indicated that inter- 
acting with the program degenerated 
to a guessing game of randomly 
assigning Ojibwa labels to drawn 
objects in the hope of matching them 
correctly. Any learning that took 
place appeared to be rote memoriza- 
tion of chance correct matchings. 
Project staff admitted that children 
were a little "confused" by the lan- 
guage lesson. 

A second Oj ibwa lesson was sup- 
posed to be in development at the 
close of the project. We do not 
know whether it was completed or 
not. 

Project staff provided some ser- 
vices to the reservation Group Home 
and to the LCO school . The LCO 
Group Home is a residential commu- 



nity for adolescents suffering from 
drug or alcohol dependency, Tlie 
project provided transportation for 
Group Home residents to Youth Center 
activities, and offered PLATO tutor- 
ing (although we do not know if 
tutoring was utilized). The Support 
Specialists provided some counseling 
to these youth. 

The project also provided trans- 
portation for LCO school children to 
proj ect activities. . 

A Human Services committee was 
established . Chaired by the Project 
Director, the committee was charged 
,^with the coordination of social ser- 
vice delivery agencies on the reser- 
vation. The project director felt 
that his time as chair of the com- 
mittee was a project contribution. 

Evaluation Results 

At the outset, it should b'e "made 
clear that we have little data by 
which to evaluate the LCO project. 
The project was still hiring staff 
as late as March 1982, and had not 
yet fully implemented one of its 
most important interventions. Youth 
Outreach. Evaluation issues took on 
secondary status to program imple- 
mentation. In addition, for reasons 
outl ined above, at the close of the 
project when summative evaluation 
data were being collected, project 
personnel were not invested in the 
project itself, much less in its 
evaluation. The project director 
was cooperative with evaluation 
efforts, but the data provided are 
insufficient to make strong conclu- 
sions. Therefore, the information 
that follows is mostly descriptive 
in nature. It is offered more to 
aid other program implementers and 
evaluators than to evaluate the LCO 
project. 
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In March of 1982 the first task 
assigned to the Youth Outreach Work- 
ers was to undertake a census of 
Youth Center users. For four weeks, 
every user of the two Youth Centers 
had to sign in. From this sign-in 
sheet a roster of Youth Center 
clients was developed. The Reserve 
Youth Center (the foremost site of 
project events) had 125 different 
users, and the New Post Youth Center 
had 81. Because the New Post Youth 
Center had little to do with the 
project, other than being staffed 
through project funds , it serves as 
a kind of non-equivalent control for 
Reserve, the project Youth Center. 

In consultation with the national 
evaluation staff, a survey was 
designed to identify the personal 
characteristics of the Youth Center 
users, and to assess their satisfac- 
tion with the Youth Centers . The 
two Youth Centers were based about 
15 miles apart. Since transporta- 
tion around the reservation is a 
substantial problem for reservation 
youth, it might be assumed that 
attendance at one Youth Center or 
the other would depend mainly on 
geography. Differences in satisfac- 
tion with or use of the Youth Cen- 
ters might be attributable to proj- 
ect effects. 

Table 1 gives some descriptive 
information on the clients of the 
two Youth Centers. The N's (the 
number of people responding to the 
survey) tells the most important 
story. Of the 125 identified 
clients of the Reserve Youth Center, 
78 were surveyed, for a response 
rate of 63%. Only 17 of 81 New Post 
youth completed their surveys, for a 
response rate of 21%. Not only is 
an N of 17 too small for results to 
be dependable, but a 21% response 
rate suggests that selection biases 
may exist. Nevertheless, Table 1 
does not suggest any major 
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differences between the youth using 
the two Youth Centers, other than 
that the New Post Youth Center might 
be serving slightly older youth. 
With the poor response rate, it is 
impossible to say with any cer- 
tainty. 

Table 2 suggests that youth visit 
the Reserve Youth Center somewhat 
more often than the New Post Youth 
Center (but the difference is not 
significant). Satisfaction with the 
youth centers does not differ much. 

Table 3 gives the stated reasons 
for visiting the youth centers. Of 
^particular interest here is the low 
percentage of youth who go the Youth 
Centers to use PLATO (by the time of 
the survey in th e summer of 1982 
PLATO had been in s t al 1 e d at New 
Post). Youth mostly attend the 
Youth Centers to do what one might 
expect: play games, visit friends, 
and get something to eat. The 
results do not support the notion 
that making computerized basic 
skills instruction available at 
recreational sites will affect many 
youth. 

Also interesting in Table 3 is 
that almost no youths report 
involvement in Youth Council Meet- 
ings. 

One possibly positive effect of 
the project^s attention to the 
Reserve Youth Center can be found in 
Table 4. Table 4 gives the place of 
residence of each survey respondent. 
Whereas all (N = 17) of the New Post 
Youth Center repondents were from 
New Post, Reserve served youth from 
all 10 of the major reservation com- 
munities. It may be that the proj- 
ect and its Outreach efforts suc- 
ceeded in involving more youth in 
Youth Center activities. Alterna- 
tively, the project may have chosen 
to operate primarily in the more 
popular center. 
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Finally, Table 5 gives descrip- 
tive information about the employ- 
ment and schooling status of the 
Youth Center users. No differences 
are significant, although somewhat 
more of the youth at New Post may be 
attending the LCO high school. 

We can make no inferences con- 
cerning the delinquency prevention 
ef f icacy of the youth centers . 
Project personnel never provided 
court contact data on the Youth Cen- 
ter users as agreed. 



What can we conclude about the 
Youth Centers ? Reserve, the primary 
project site, seems to serve youth 
from a wider geographical area than 
the New Post Youth Center, which 
received less attention from the 
project. This may or may not be a 
project effect. According to the 
project director, the Reserve Youth 
Center was more active than New Post 
prior to the start of the project. 
That was why Reserve was chosen as 
the focus of the project. 

To be fair, more important than 
any differences between the youth 
centers is their fundamental simi- 
larity: they were open for busi- 
ness. They would have been closed 
or opened on a much curtailed sched- 
ule without grant funding. Thus, 
their availability to youth was a 
project success. For theoretical 
reasons, however, their usefulness 
in preventing delinquency is ques- 
tionable. 

Youth Outreach 

We have no direct way of evaluat- 
ing the effectiveness of the Youth 
Outreach component, including the 
Youth Outreach Workers, and the Sup- 
port Specialists. Yet, three com- 
ments are in order: 



1- As just discussed, the 
greater geographical distribution of 
the Reserve Center's users may be 
due to Outreach efforts, since Sup- 
port Specialists and those Youth 
Outreach Workers that had been hired 
were based there. The evidence for 
this in only circumstantial. 

2 . The Youth Center survey 
administration was the single most 
important task of the Youth Outreach 
Workers, and it .was performed 
poorly. This suggests that high 
school students require careful 
supervision to carry out such a 
field study. Similar supervision 

.^would probably be required for the 
other duties, including peer coun- 
sel ing and family visits, with which 
they were charged. 

3. Support Specialists may 
require more preparation in drug 
dependency counseling. Because they 
were charged with supervising the 
Youth Outreach Workers in the Youth 
Center survey administration, which 
did not go well, they may have 
required more preparation in project 
management or supervision as well. 

Alternative Education 

We were not provided with infor- 
mation needed to identify Youth 
referred to "Alternative Education" 
during the 1981-82 school year. 
Both the principal of the high 
school and the project director 
indicated that the program did not 
work: Youth suspended from high 
school for nonattendance did not 
come to the Youth Center for basic 
skil Is tutoring , or counsel ing in 
drug dependency. 

For descriptive purposes. Table 6 
gives characteristics of the refer- 
rals to Alternative Education during 
the spring of 1981. 
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Iqbi able 8tudont8 who are using 
PLATO* Neither is there any rela- 
tionahip between PLATO uoe and any 
meaaure of peychological adjustment. 
We find no evidence here that PLATO 
uae increased the welf-esteem of low 
skilled learners. 

Nothing in those results suggests 
PLATO instruction had a measurable 
effect on anything. 

Did the Program M eet Its Goals and 
Objectives ? 

Probably not* We cannot assess 
the project's impact on delinquency. 
Self-report delinquency was assessed 
in the LCO high school, but the re- 
sponse rate for students in the pub- 
lic school was extremely low, and 
besides, the Youth Centers were the 
prime project intervention. Since 
the court contact data was never 
delivered, we are blind to the 
effects of the Youth Centers on 
delinquency. There is no reason to 
believe they had an effect on delin- 
quency • 

We cannot say whether the project 
affected dropout from the LCO 
school. Since the intervention in 
the school was weak (paying for com- 
puters that were already there) we 
do not know why dropout should have 
been affected. We do know that the 
"alternative education" inteirven- 
tion— referral to the Youth Center 
for tutoring after suspension — was 
not successful. No one came. 

Similarly, we cannot evaluate the 
project's effects on suspensions and 
absenteeism from the LCO school , or 
on the self-esteem of LCO youth. 
Sane weak evidence suggests that 
youths attending the LCO school are 
suspended less than they might be 
were they attending the public 
school. The project certainly did 
provide recreational opportunities 
for youth. 



Recommendations 

1. Proiects should not be funded 
that do not have a clearly specified 
theore tical base supporting a 
rational , planned set of interven - 
tions . Especially given the short 
t ime per iod ( two y ear s ) by the end 
of which projects are required to 
have demonstrated progress toward 
goals and objectives, programs that 
are not able to hit the ground run- 
ning are faced with an impossible 
task. 

2. Proiects should not be 
allowed to expend funds on "scatter - 
shot" interventions . The LCO proj- 
ect spent time, money, and energy on 
a shopping basket of youth interven- 
tions. It would have better concen- 
trated its efforts on a smaller sub- 
set of interconnected interventions. 
The Youth Outreach program was an 
attempt to do this,_ but ^ there 
remained insufficient time for" its 
implementation. 

3. Personnel implementing an 
intervention should have the requi - 
site training and experience . Only 
when counseling is done well is 
there much hope of positive effects. 
Indeed, the evidence suggests that 
counseling can be of no help in pre- 
venting delinquency (Wright and 
Dixon, 1977), and may have negative 
effects (McCord, 1978; Cook, 1983) 
that should be carefully guarded 
against. Under the circumstances, 
state-of-the art counseling inter- 
ventions implemented by well-trained 
professionals are perhaps worthy of 
test, but the, efforts of amateurs 
are unlikely to be fruitful. 

^* The requirement of rigorous 
evaluation should be made clear 
prior to funding , and a clearly spe - 
cified experimental or strong quasi- 
experimental evaluation design 
should be a requirement for funding . 
Federally funded crime prevention 



-428- 



314 



LCO 



projects should come to anticipate trol groups and the collection of 

careful evaluations involving con- adequate outcome data. 
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Table 1 



Descriptive Statistics 
on Youth Center Patrons, Spring 1982 



Reserve: New Post: 

Proiect Youth Center Comparison Youth Center 
Characteristics M SD N M SD N 



Age in years 
Male 

Holding high 
school diploma 



16.17 5.77 76 

63% — 78 

31% — 78 



18.50 5.61 17 
63% ~ 17 

25% — 17 



Table 2 



Frequency of Attendance and Satisfaction 
with Youth Centers, Spring 1982 



Reserve: New Post: 

Project Youth Center Comparison Youth Center 
Patron's report of M SD N M SD N 



Weekly visits 

to youth center 3.93 1.75 75 3.06 1.71 17 

Satisfaction with 
youth center 2.81^ .68 79 2.94 .43 17 



On a four-point scale ranging from "Very Satisfied" (4) to 
"Very Dissatisfied" (1). 
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Table 3 



Reasons for Visiting Youth Center, Spring 1982 
(Column Percent) 



Reason 



Reserve: Project 
Youth Center (n=79) 
% Affirmative 



New Post: Comparison 
Youth Center(n=17) 
% Affirmative 



Games (Pinball, 
video, pool) 

Food 

Social contacts 

Use PLATO 

Youth Council 
meetings 

Other 



90 
68. 
66 
8 

1 
9 



94 
47* 
71 
12 

0 
0 



^p<.10 
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Table 4 



Residence of Youth Center Patrons, Spring 1982 
(Column Percent) 



Community of Reserve: Project New Post: Comparison 

Residence Youth Center (n=78) Youth Center (n=17) 



Bacon Square 


4 


0 


Chief Lake 


11 


0 


Dry Town 


1 


0 


K-Town 


6 


0 


New Post 


4 


100 


Reserve 


48 


0 


Round Lake 


6 


0 


Signor-Couderay 


9 


0 


Six Mile 


4 


0 


White Fish 


4 


0 


Other 


4 


0 



Table 5 



School and Work Status of Youth Center Patrons 
Responding to, "What are you Doing Mostly?", Spring 1982 

(Column Percent) 



School and Work Reserve: Project New Post: Comparison 
Status Youth Center(n=78) Youth Center(n=17) 



Attending LCO 
high school 

Attending LCO 
elementary school 

Attending Hayward 
public high school 

Attending Hayward 
public elementary 
school 

Attending 
vocational school 

Attending G.E.D. 

Attending college 

Working full or 
part time 

Doing tribal 
volunteer work 

Unemployed 

Other 



23 



1 
1 
1 

24 

1 
15 
23 



41 
6 
6 



0 
0 
0 

24 

0 
18 
6 
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Table 6 



Descriptive Statistics on Referrals to 
"Alternative Education", Spring 1981 



Student Characteristics 



Grade point average, 
Spring 1981 

Hours PLATO use. 
Spring 1981 

Absences, Spring 1981 

Disciplinary referrals. 
Spring 1981 



M SD • N 

2.00 0.00 5 

7.05 5.17 11 

9.63 3.07 8 

1.09 .70 11 



Note: Six of the 11 referrals withdrew dur- 
ing the Spring of 1981. Three of these sub- 
sequently re-enrolled the following year, 
and one more dropped. At the end of 1982, 7 
of the 11 1981 referrals were still enrolled 
in school. None had graduated. 
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Table 7 



Comparison of Self-report Measures of 
Indian Youth Attending the LCO High School and the 
Sawyer County (Hayward) Public Schools, Spring 1982 



LCO Public School 

Questionnaire Measures M SD N M SD N 



School variables 



Attachment to school 




9 9 


77 


7 n 


• JU 


zU 


School rewards . 


.43 


.33 


78 


.10 


.22 


22*** 


kJ\^ll\J\J^ ^ UIl ^ O lllUCil I. o 




1 7 


7 G 


. U/ 


.19 


22 


Parental emphasis on education 




.30 


76 


.30 


.21 


22*** 


School effort 


.55 


.29 


77 


.51 


.28 


24 


Negative peer influence 


.29 


.19 


79 


.27 


.22 


25 


Victimization 


.10 


.15 


77 


.06 


.10 


21 


Involvement 


.20 


.16 


77 


.15 


.14 


23 


Reading ability 


1.92 


1.59 


79 


1.85 


1.64 


26 


Psvcho-social variables 














Attachment to parents 


.42 


.24 


79 


.55 


.26 


25*** 


Belief in rules 


.61 


.28 


76 


.71 


.28 


22 


Positive self-concept 


.69 


.17 


72 


.70 


.17 


21 


Rebellious autonomy 


.86 


.23 


75 


.71 


.36 


22** 


Interpersonal competency 


.77 


.25 


73 


.76 


.27 


22 


Internal control 


.44 


.24 


75 


.42 


.28 


22 


Alienation 


.27 


.24 


76 


.24 


.26 


21 


Practical knowledge 


1.34 


.40 


76 


1.22 


.61 


22 


Invalidity 


.14 


.18 


77 


.05 


.12 


22** 


>elf-reported behavioral variables 














School non-attendance 


1.29 


.66 


78 


1.00 


.82 


25 


Delinquency — total 


.28 


.19 


52 


.14 


.15 


25*** 


Serious delinquency 


.14 


.17 


52 


.07 


.14 


25* 


Suspensions 


.62 


1.98 


79 


1.85 


3.57 


26** 


Drug use 


.60 


.32 


51 


.28 


.31 


25** 



*p<.10 
**p<.05 
***p<.01 
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Table 8 



Correlations between 1982 Hours of Plato Use 
and Measures of Academic Achievement, School 
Attachment, Psychological Adjustment, and 
Delinquent Behavior 



Measures 



N 



Academic achievement 

Spring 1982 GPA^ 
School effort 
Absences^ 

Withdrawals from school^ 
Attachment to school 



.06 
.09 
.29* 
.16 



73 
77 
89 
95 



Attachment to school 
School rewards 
School punishments 

Psychological adiustment 

Positive self-concept 
Alienation 
Internal control 
Interpersonal competency 

Delinquent Behavior 

1982 Discipline referrals^ 
Self-reported delinquency — total 
Self-reported serious delinquency 
Self-reported drug use 



.09 
.12 
.03 



.09 
,10 
.07 
.18 



.28* 
.04 
.12 
.05 



77 
78 
78 



72 

76 
75 
73 



95 
52 
52 
51 



School-based data; other measures are SAES ques- 
tionaire-based. 



«fp<.05 
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Plymouth Alternative Education Project: Second Interim Report 

S. Cook 



Abstract 

The Plymouth Alternative Education Project operated Student Ser- 
vice Centers, Student Activities Centers, and an out-of-school 
Learning Options program primarily for two high schools and two mid- 
dl e school s . These interventions provide educational service s , 
counseling, and recreation for students with disciplinary and atten- 
dance difficulties in this predominantly working and middle-class 
white community . Interim evaluation results imply that the high 
school Student Activities Center is being implemented with care, and 
has some promising positive effects on participants (increased aca- 
demic performance , decreased al ienation, and increased practical 
knowledge). Interim evaluation results raise some questions about 
the Student Service Center implementation, and suggest that some 
unexpected negative effects of this counseling intervention may be 
occurring. The Learning Options program, which appears to be vigo- 
rously implemented, did not participate in an outcome evaluation. 



The Proiect 

The Plymouth Alternative Educa- 
tion Project is one of several proj- 
ects funded by the federal Office of 
Juvenile Justice and Delinquency 
Prevention (OJJDP) as part of its 
Program in Delinquency Prevention 
through Alternative Education. The 
Plymouth project operates in two 
large high schools that share the 
same physical plant, and in two 
separated middle schools. All are 
located in a mostly white working, 
and middle-class suburb of Detroit, 
Michigan. The project seeks to pro- 
vide services to students exhibiting 
truancy, disruptive behavior, or 



poor academic performance, and who 
are considered at risk for drop-out 
and delinquency. The project is 
composed of four discreet treatment 
programs : 

1. The high school Student Ser- 
vices Center is a daily semester- 
long class for which students 
receive credit toward graduation. 
Treatment involves four major inter- 
ventions : (a) f acil itated group 
peer counseling, using the students' 
day-to-day problems as the source of 
group discussion, coupled with 
affective educational training in 
transactional analysis and interper- 
sonal communication skills, 
(b) close weekly monitoring of the 
students' academic progress in their 
other classes, (c ) individual coun- 
seling outside of the class period, 
and (d) a high level of counselor- 
initiated parent contact. 

2. The high school Student Activ- 
ities Center has three distinct 
interventions: (a) a competency- 
based remedial writing and study 



This report covers roughly the 
period September 1981 through August 
1982. For an account of the project 
during its first year see Carlton 
(1982). 
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skills class for students showing 
poor academic performance, (b) a 
supervised setting for student ini- 
tiated activities such as clubs and 
discussion groups, and (c) an organ- 
ized intramural program targeted at 
students who are uninvolved in con- 
ventional school extra-curricular 
activities. 

3. The middle school Student 
Services Center is similar to the 
high school program of the same 
name. There are three major differ- 
ences between the two interventions, 
all of which are due to the matura- 
tional differences' of the respective 
cli<=?nt populations: (a) students 
mecu weekly on an individual basis 
with their counselor for ten weeks 
prior to the formation of groups; 
this aids in developing bonding to 
the counselor so that the counselor 
has more leverage over negative peer 
influence effects that could appear 
in this age group; (b) the middle 
school program is team taught; two 
counselors facilitate the group dis- 
cussions 'SO that they have more 
direct control over the group pro- 
cess; and (c) the program is more 
structured, relying less on the stu- 
dents themselves to bring up prob- 
lems as a source of group discus- 
sion. 

4. Learning Options (LO) is a 
completely self-contained alterna- 
tive school for highly disruptive 
students from both the high and mid- 
dle schools (though most program 
participants in 1981-82 were middle 
school students ) . It is character- 
ized by a high degree of individual- 
ized instruction, tutoring , group 
and individual counsel ing , progres- 
sive attainment of responsibil ity , 
student input into rules and disci- 
plinary practices, and parental 
involvement in their children's 
schooling. 

Besides the above programs, proj- 



ect staff also seek to influence the 
general school cl imate of the four 
project schools by serving as infor- 
mational sources for alternative 
discipline procedures, and by sit- 
ting on appropriate building and 
district-level committees concerned 
with discipl ine , attendance , and 
school rules. 



Program Changes During 1981-82 

The only major changfe in the -high 
schooL SSC between the 1981 and 1982 
school years was the decrease in the 
amount of time that the class was 
, team taught. In previous years, the 
project co-directors had often team 
taught the class, particularly when 
students were having difficulty, or 
some special problem came up. 
Apparently due to time spent engaged 
in evaluation activities, the SSC 
counselor was_force^ to_ work__ alone 
in the SSC class more than antici- 
pated. 

In the SAC two changes were made 
in the program during the year. 
First, a pre- and post-testing 
procedure for the remedial writing 
class was revised to make it more 
sensitive. It was changed from a 
five-part to a seven-part examina- 
tion. The SAC instructor devoted a 
great deal of time to developing an 
adequate diagnostic and evaluative 
instrument — one that paid handsome 
dividends (see Evaluation Results, 
below) • 

A second change in plans for the 
SAC was the emphasis that was placed 
on the remedial writing class for 
the SAC participants . In fact , 
"Student Activities Center" was 
somewhat of a misnomer for the 
intervention during the 1981-82 
year. The primary intervention of 
the SAC was, in fact , the writing 
class, and student activities took a 
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A more extensive version of the 
scale has been developed for 
1982-83, and will be included in the 
regular version of the SAES student 
questionnaire. 

The G.P.A. measure reported in 
Table 2 is only a weak measure of 
achievement. This is because it was 
the G.P.A. for the entire academic 
year. Therefore, any effects of the 
second semester treatment will be 
watered down by the first semester 
pre-treatment G.P.A. All reported 
high school G.P.A. data suffers from 
this defect. 

Table 1 shows that out of the 14 
measures of the SSC goals, four are 
significantly different for treat- 
ment and control group students. 
Only one of those differences favors 
the treatment group: Treatment stu- 
dents attempted significantly more 
classes than the control group. 
This, in effect, is an implementa- 
tion measure. One of the program 
objectives is to move the students 
toward graduation, and SSC partici- 
pants are required to take a full 
load of classes, including the SSC 
itself, for which they receive 
credit toward graduation. Impor- 
tantly, the treatment students do 
^ot pass any more classes than the 
control students* 

The SSC results in three negative 
effects on goal measures. The SAES 
measures of Interpersonal Competency 
and Parental Emphasis on Education 
both significantly favor the control 
group. The treatment students also 
report more drug use than the con- 
trol students. 

The SAES measures of Alienation 
and Interpersonal Competency are 
perhaps the most direct SAES meas- 
ures of the attitudinal and intra- 
personal variables that the SSC 
seeks to affect. In this data. 
Alienation shows no difference, and 



Interpersonal Competency shows a 
negative effect of the treatment. 

The negative effect on drug use 
is particularly disturbing, since it 
suggests that far from decreasing 
the drug use of their clients, the 
project is increasing SSC students' 
involvement in drugs. 

Turning to Tabl e 2 , of the 12 
non-redundant measures of the SSC's 
objectives, there are two dependably 
and two marginally ' significant 
effects. Two of these effects favor 
the treatment, and two the control. 
Treatment students are slightly less 
likely to be suspended out of 
school, and are also somewhat less 
1 ikely to voluntarily withdraw from 
school. These are important, if 
weak, positive program effects. The 
controls, on the other hand, skip 
class less, and report themselves to 
be more involved in conventional 
(essentially school related) activi- 
ties. No other differences, includ- 
ing four other relevant measures 
(see the bottom of Table 2) approach 
significance. 

Several l^po theses were advanced 
by the project co-directors to 
account for these results. They are 
explained more fully below, but one 
included the observation that posi- 
tive benefits of the program may be 
of a more long-term nature; students 
may initially have some negative 
affect as they are forced to con- 
front themselves and their problems 
in the group context, but that the 
sel f-1 earning that occur s , al ong 
with the support provided by project 
staff, should better equip the stu- 
dents to cope with the large, imper- 
sonal school environment. In order 
to examine this hypothesis , a sepa- 
rate series of analyses of variance 
were run looking just at the first 
semester treatment and control stu- 
dents. The treatment students 
should have had an extra semester 
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for the longer term effects of the 
treatment to be manifested. Because 
of the decreased number of experi- 
mental participants, this analysis 
will be less sensitive than the 
whole year analysis. 

Table 3 gives the results for all 
significant effects and nearly sig- 
nificant effects, for the same out- 
comes listed on Tables 1 and 2. 
Also listed are the results of an 
analysis of the Invalidity scale, a 
measure of truthfulness in report- 
ing. There are six significant 
effects, all of them indicating a 
negative effect of the treatmeiit. 
First semester treatment students as 
compared to controls reported less 
Interpersonal Competency, lower 
Self-esteem, more Rebellious Auto- 
nomy, less Involvement in conven- 
tional activities, lower Attachment 
to School, and less Parental Empha- 
sis on Education. No other measures 
approached significance. 

The hypothesis that the SSC stu- 
dents would exhibit increased posi- 
tive programmatic effects over time 
is not supported. If anything, the 
negative effects on the interper- 
sonal and intrapersonal variables 
measured by the SAES increase over 
time. 

Potential explanations of the 
negative treatment effects . To what 
do we a t tr ibut e th e sh or t- and 
long-term effects of the SSC treat- 
ment? Analysis of program implemen- 
tation, and discussion with program 
personnel suggest eight possible 
explanations for the combination of 
effects found: 

1. Composing the groups entirely 
of students with academic and beha- 
vioral problems may have resulted in 
non-conventional group norms. This 
could have produced in- and out-of- 
school friendship cl iques that were 
predominantly non-conventional in 
nature. 



The SAES questionnaire includes a 
measure of Negative Peer Influ- 
ence — the extent to which the res- 
pondent's peers get in trouble and 
encourage the respondent to be 
anti-conf orming . Analyses of vari- 
ance of this measure for tlie first 
semester treatment and control stu- 
dents shows that there is a very 
weak trend toward treatment students 
(M = .48, N = 17) reporting more 
negative peer influence than control 
students (M = .40, N = 22; p =.19). 
For the comparison of all experimen- 
tal students for the whole year, we 
find a similar, but even weaker pat- 
tern of effects. The treatment 
^group (M = .46, N = 46) reports very 
si ightly more negative peer influ- 
ence than does the control group (M 
= .41, N = 42; p = .27). This dif- 
ference is trivial and non-signif i- 
cant . 

Evidence more strongly supporting 
a negative^ peer" influence "effect 
comes from the difference in drug 
use and class attendance between 
treatment and control s in the year 
sampl e . Adole scent drug use is 
highly related to peer use and may 
be subject to peer influence (Kan- 
del, Kessler, & Margulies, 1978). 
The treatment students may be influ- 
encing each other to increase drug 
use. Likewise, the increased class 
cutting is consistent with the nega- 
tive peer influence hypothesis. lu 
al 1 , the evidence for a negative 
peer influence effect is equivocal. 
Project personnel should neverthe- 
less be attentive to its effects. 

2. A second possible explanation 
for the negative effects of the SSC 
treatment lies in the possibiltiy of 
a counselor-validated norm that, 
"The school stinks, but you need 
that piece of paper." A primary 
goal of the Plymouth project is to 
increase the graduation rates of 
potential drop-out students. One of 
the ways that the SSC addresses this 
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goal' is to validate the students' 
perceptions concerning the imper- 
sonal nature of the school and the 
lack of caring of some of their 
teachers and parents, while at the 
same time emphasizing to the stu- 
dents how important the high school 
diploma is. The students are taught 
to cope with secondary education, 
and survive in the building, but not 
necessarily to enjoy school • This 
explains how the students can become 
less attached to school , decrease 
their involvement in school activi- 
ties, and skip class more often, and 
at the same time reduce their rate 
of voluntary withdrawals* Given the 
strong relationship between attach- 
ment to school and graduation rates 
(Bachman, O'Malley, & Johnston, 
1978; Gottfredson, 1982) this is 
a risky approach to the problem of 
drop-out* This is particularly true 
since the students are attending 
class significantly less than the 
controls, despite programmatic 
efforts to closely monitor atten- 
dance. 

3, An explanation for the con- 
sistent negative effects on the 
Parental Emphasis on Education 
self-reports comes from the data 
found in Table 4, Table 4 shows the 
number of parent phone calls, parent 
visits, and parent meetings recorded 
for each of the four Plymouth proj- 
ect treatments. Collapsing parent 
visits and parent meetings, it is 
apparent that, although parents are 
highly encouraged by program person- 
nel to become involved in their 
children's education, and are conti- 
nually invited for conferences and 
program activities, the high school 
SSC treatment parents average less 
that 1.5 personal visits per semes- 
ter* Throughout the semester, the 
students are behavioral ly reminded 
that their parents do not , in fact , 
support their educational efforts. 
They are given clear evidence that 
their parents do not care enough to 



attend program functions. Whether 
this is a long-term negative effect 
of the program can only be known 
through follow-up. 

.4. A hypothesis advanced by the 
project co-directors for the nega- 
tive self-report measures (excluding 
self-reported drug use) is that the 
positive intra-psychic effects of 
the . program will only occur over 
time. This was the reason that 
separate comparisons were made of 
the first semester only treatment 
and control groups. That analysis 
indicated, that if anything, the 
negative attitudinal effects 
increased over time. Yet, it is 
still possible, although unl ikely, 
that a still longer follow-up period 
will be necessary to demonstrate 
positive effects. It must be said 
that at the current time the weight 
of the evidence argues precisely 
opposite to a delay^ _of _pqsi^tiye 
effects explanation. 

5. Another hypothesis concerning 
the self-report measures is that the 
treatment students responded more 
honestly. A large part of the 
affective education instruction pro- 
vided in the SSC class involves 
teaching students to honestly evalu- 
ate their intrapersonal feelings and 
responses. Students are encouraged 
to openly express their feelings in 
the class. Indeed, one of the pri- 
mary reasons that it became diffi- 
cult to design a middle school SSC 
that par al lei ed the high school 
model was that middle school stu- 
dents could not as easily trust one 
another to keep confidences so that 
close personal feel ings could be 
expressed without fear of ridicule. 
The project co-directors argued that 
this intrapersonal reflection and 
emphasis on honestly would contri- 
bute to differences between treat- 
ment and control students through a 
two stage process: (a) because of 
the emphasis on examination of them- 
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selves and their problems students 
would become more aware of the prob- 
lems they have — they would be carry- 
ing less "false autonomy," and 
(b) they would therefore be more 
likely to report honestly about 
their personal feeling and behaviors 
when responding to the question- 
naire. Tliese are two subtly differ- 
ent processes, but they would com- 
bine to make the treatment group 
more likely to report negative 
events on certain measures, even if 
they did not actually differ; in 
fact, they may not be as bad off as 
the control students. Ironical ly, 
in the view of the project co-direc- 
tors such "negative" self -report 
effects of the treatment could actu- 
ally be positive treatment effects. 

It is difficult to evaluate the 
accuracy of this claim. Some data 
is available from the SAES Invalid- 
ity scale. Invalidity is a measure 
of truthfulness in reporting and 
contains items such as "I read sev- 
eral whole books every day." 
Although this measure was designed 
to detect students who were not 
responding seriously and honestly to 
the survey , it shows no difference 
between the first semester treatment 
and control groups (see Table 3 ) . 
But , the argument advanced is that 
the control students may not be as 
"self-aware" as the treatment stu- 
dents, so the question is really not 
one of lying on the questionnaire. 
Therefore, the invalidity scale only 
obliquely addresses the "honest" 
reporting issue. 

On the other hand, the "self-a- 
ware/honesty" explanation does not 
explain the differences in self-re- 
ported drug use ; the measure is a 
factual report of use that does not 
rely on being more self-aware for 
more accurate reports. Particularly 
since there were no differences on 
the overal 1 del inquency sel f-re- 
por ts , it is difficult to hypothe- 
size that , the treatment students 



were responding more honestly to the 
behavioral measures. Why should the 
students honestly report their ille- 
gal drug use , and falsely report 
other delinquent acts? 

Obviously, differences in self- 
report style cannot account for the 
negative program effects on class 
attendance where school records were 
the source of the measures. 

6 . Another explanation for the 
pattern of negative effects is that 
r andom iz a t ion br oke down , and tha t 
the experimental and control groups 
differed at the outset of the evalu- 
ation. Table 5 gives the results 
for analyses of variance comparing 
all program treatment groups and 
their respective controls on affec- 
tive pretest measures gathered prior 
to the experiment . Because many of 
the students failed to adequately 
complete the pretest, N's for these 
analyses are very low. Neverthe- 
less, there is no indication from 
these data that the high school SSC 
students differed at the beginning 
of the treatment period. Only one 
comparison out of nine approaches 
significance, and this is well 
within the bounds of chance. 

7. A seventh possible explanation 
for the results is that the negative 
self-report effects are real, and 
that they may be the result of 
inconsistent implementation of pro- 
gram components. Unfortunately, we 
only have two sources of implementa- 
tion data. One is records of parent 
contacts, the other is the actual 
time, in days, spent in the treat- 
ment program. The correlations of 
parent contacts and time in treat- 
ment with each other and with al 1 
outcome measures were calculated. 
None of these correlations was sig- 
nificant. Table 6 shows the corre- 
lations between time in treatment 
and parent contacts. It is apparent 
that there is no relation between 
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the amount of time the treatment 
students are in the class, and the 
number of parent contacts that are 
made. If contact with the students' 
parents is an integral part of the 
treatment process, then at least 
this component of the intervention 
is not being systematically applied. 

Related to implementation issue 
is the significant increase in class 
cutting shown by the treatment stu- 
dents . Because class attendance is 
an important target behavior for the 
program , it is obvious that program 
implementation is breaking down. 
One of the very behaviors that is 
specif ical ly addressed is moving in 
the wrong direction. 

8. A final possible explanation 
for the negative self-report effects 
could lie in differential response 
to the SAES , questionnaire. Table 7 
lists the reponse rates for the var- 
ious Plymouth pro j ect sampl ing 
groups. Fewer of the high school 
SSC control than treatment students 
completed the questionnaire. Since 
the students who are not surveyed 
may show poorer outcomes than more 
cooperative students, some of the 
negative effects may be due to the 
differential survey response rates. 
Of course, to explain the self-re- 
ported drug use results one would 
have to argue that the non-surveyed 
control s differed from the surveyed 
treatment students in their drug 
use , but not in their del inquency . 
In any case, the differential survey 
rates make ev ident the need for 
strengthened efforts to administer 
the 1983 SAES student questionnaire 
to control group students to achieve 
high response rates for both treat- 
ment and control groups. 

Examining the entire pattern of 
results, and the possible plausible 
and implausibl e explanations for 
them forces us to conclude that the 
jury is still out on the high school 



SSC. The sel f-report results are 
seemingly negative (especial ly for 
drug use), and class attendance is 
negatively affected, yet there is a 
strong positive trend in the whole 
year sample in the key behavioral 
measures of voluntary withdrawal s 
and out-of-school suspensions. We 
will have to wait for replication 
results and an examination of new 
data with higher control group res- 
ponse rates to reach a final conclu- 
sion concerning the efficacy of the 
high school Student Services Center 
model .. 

One closing point should be noted 
concerning the high school Student 
Services Center : the program is 
very popular in the school system. 
It has been funded in the district 
for nine years, and will be sup- 
ported when federal funding is no 
longer available. School system 
administrators, at J-east ,_yiew_ the 
program favorably. When confronted 
with the interim evaluation results, 
the typical administrator ''s response 
reportedly is "Johns Hopkins must 
not be measuring the right things." 
Although we can haf dly imagine a 
longer questionnaire, or more exten- 
sive data collection, the project 
has cooperated in helping us design 
"affective education" measures. 
They have been incorporated into the 
1983 version of the School Action 
Effectiveness Study survey. Further 
speculations await better data. 

High School Student Activities Cen - 
ter 

A true experiment was carried out 
during the second semester of the 
1981-82 year to evaluate the high 
school School Acitivities Center 
writing class (SAC). Table 8 gives 
the results for analyses of variance 
comparing random treatment and con- 
trol students on measures of the 
sac's goals. 



-446- 



332 



'"lymouth 



The Capitalization, Punctuation, 
Words, Sentences, Sentence Struc- 
ture, Study Skills, and Essay meas- 
ures are all derived from pre- and 
post-tests measuring the writing 
skills covered during the siting 
class. Of the six test component 
comparisons, (the Essay comparison 
is based on too few persons for 
interpretation) three are signifi- 
cant, and one shows a marginally 
significant trend. In all cases the 
treatment students perform better 
than the control students. It is 
apparent that treatment students are 
improving their writing skills. 

Four other measures of the SAC's 
goals are also significant. SAC 
clients attempt more classes than 
the control students (and the per- 
centage of classes they pass 
approaches significance, p = .12). 
SAC students report themselves to be 
less alienated, and to have greater 
practical knowledge. Although the 
SAC students report themselves to be 
receiving significantly higher 
grades, the results of the analysis 
of the actual grade data do not sup- 
port the self-reports. As noted 
earlier, however, the G.P.A. data 
are for the entire academic year, 
and the experimental students were 
only in the program during the sec- 
ond semester. Any program improve- 
ments would be diluted by the first 
semester pre-treatment data. The 
fourth effect is a non-significant 
trend for SAC students to report 
that their parents put less emphasis 
on their education. 

Table 9 gives the results of 
analyses of variance comparing the 
treatment and control students on 
measures of the SAC's objectives. 
None of th e me a s ur e s th a t do not 
overlap with SAC goals are statisti- 
cally significant (Alienation is 
discussed above). 

The experimental evaluation of 



the SAC writing classes suggests 
three conclusions: 

1 . The students are improving 
their writing skills, and may be 
increasing their overal 1 academic 
performance , at least so far as it 
is measured by self-reported grades. 
Follow-up of these students should 
result in better information con- 
cerning general academic improve- 
ments. 

2 . Two psychosocial devel opment 
outcomes favor the SAC students: 
Alienation and Practical Knowledge . 
We do not know whether these effects 

^ are a direct result of the class and 
the counsel ing that accompanies it, 
or an indirect effect of increased 
academic success. Since the SAC and 
SSC students appear to be very simi- 
lar by the pretest measures (see 
Table 5), and the SSC program is 
expl icitly a counsel ing program, 
while the SAC is primarily a reme- 
dial academic program, one cannot 
but be struck by the possibility 
that increased academic achievement 
results in better "psychological 
health" — and that the SAC is a more 
fruitful intervention than the SSC. 

3. The trend toward the SAC pro- 
ducing lower self-reports of Paren- 
tal Emphasis on Education is inter- 
esting because it parallels results 
found for the high school SSC. It 
was argued for the SSC that program 
personnel seek to involve parents in 
their children's education, but 
report that they are largely unsuc- 
cessful. This may serve to rein- 
force for the students the knowledge 
that their parents do not care about 
their educational progress. A simi- 
lar process may be operating in the 
SAC. The control students in both 
the SSC and SAC may be in blissful 
ignorance of their parents Tack of 
concern about their education. 
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4, A fourth conclusion is that 
the SAC seems to be well imple- 
mented. This conclusion flows from 
two pieces of data. The first is 
the observation that the SAC 
instructor has spent a great deal of 
time developing curriculum materials 
and pre- and post-tests to monitor 
the progress of his students. It is 
significant that he has revised his 
pre- and post-tests. This indicates 
that he is paying attention to the 
data he receives from his test, and 
is responding accordingly. This is 
a positive sign for any program 
development effort, 

A second pi&ce of implementation 
data can be found in Table 10, This 
table shows the correlations between 
parental contacts and time in the 
SAC class — measures of implementa- 
tion — with selected pretest and out- 
come variables. Parental contacts 
are highly correlated with several 
of the outcome variables, notably 
those having do with discipline 
infractions, and academic perfor- 
mance. Particularly strong (r = 
•60) is the correlation between 
classes missed and parental con- 
tacts; the SAC instructor is notif- 
ying parents when their children are 
not attending class. Also strongly 
correlated with parent contacts is 
the Expression scale measured at 
pretest; tht more expressive the 
student , the more the parents were 
contacted. This pattern of correla- 
tions is in TTir.rked contrast to that 
found in th SSC data in which the 
parental itacts were not corre- 
lated w ■ '.li c.ny other measure. Table 
10 al so shows that time in the SAC 
class is unrelated to parental con- 
tacts, so, like the SSC, parental 
contacts are not uniformly distri- 
buted over time. On the other hand, 
relationships between parental con- 
tacts and outcome variables show 
that the SAC instructor seems to be 
responding systematically to the 
behavior of the students. 



The Middle School Student Services 
Center 

Early in the 1981-82 year an 
attempt was made to implement a true 
experiment in the middle school Stu- 
dent Services Center (SSC). This 
de s ign br oke down when th er e we r e 
insufficient referrals to enable 
students to be randomly assigned to 
treatment and control groups . 

As a fallback design, the project 
co-directors identified* a non-equi- 
valent control group of students in 
the school , and administered the 
project pretest to them. Table 5 
^gives analyses of variance results 
comparing the SSC participants to 
their non-equivalent controls on 
pretest self-report scales. Of the 
nine comparisons, five are signifi- 
cant. Three of the pretest measures 
(Alienation, Attachment to school, 
and Expression) favor the control 
group, and two^ (Bel lelf Tn' the" rules 
and Interpersonal Competency) favor 
the treatment group. It is puzzling 
that some of the measures favor the 
treatment group, and some the con-, 
trol, but the consequence of the 
gross differences between the groups 
is that it will be very difficult to 
explain any differences between 
groups on outcome measures because 
it will be difficult to disentangle 
treatment effects from pretreatment 
differences. 

Tables 11 and 12 give the results 
for analyses of variance comparing 
the treatment and non-equivalent 
control groups on measures of the 
middl e school SSC goal s and ob j ec- 
tives. Of thirty- three outcome 
measures, fourteen show statisti- 
cally significant differences 
between the groups and five others 
show strong trends. In all cases, 
the treatment group shows poorer 
outcomes than the control group. 
Recall, however, that it is probable 
that the groups differed before 
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treatment. The analyses imply that 
the treatment does not change the 
behavior of referred students to 
match that of non-referred students. 
(Analyses of covariance controlling 
for pre-existing differences were 
uninterpretable; too few students 
completed both pre- and post-tests 
to allow for meaningful analyses.) 

One positive note for the program 
may be found in Table 13. This 
table lists selected correlations 
between parent contacts and time in 
the SSC with the various outcome 
measures. The table indicates that 
the counselors in the SSC are 
responding to misbehavior and 
truancy by contacting parents. 

Learning Options 

We were unsuccessful in eliciting 
an agreement from Learning Options 
personnel to implement a true exper- 
iment or any other satisfactory sum- 
mative evaluation. Since Learning 
Options serves adolescents with sev- 
ere behavior problems, identifying a 
meaningful non-equivalent control 
group is difficult. Therefore, we 
are left with no control group and 
no outcome evaluation of the Learn- 
ing Options program. This is unfor- 
tunate, in that the Learning Opti/ons 
program appears to be a theoreti- 
cally based and well-implemented 
attempt to deal with problem stu- 
dents in an alternative education 
format. 

For descriptive purposes Table 14 
gives the means and standard devia- 
tions of rel evant outcome measures 
for the Learning Options students. 

Other Project Accomplishments 

A major accompl ishment of the 
Plymouth Alternative Education Proj- 
ect in 1981-82 was the implementa- 
tion of a new Plymouth-Canton high 



school attendance and withdrawal 
policy. The old attendance and 
withdrawal policy was that students 
were automatical ly withdrawn from 
the high school if they accumulated 
eight unexcused absenses. The proj- 
ect co-directors, while serving on 
the high school's Attendance Policy 
Committee, were instrumental in get- 
ting this pol icy changed to one in 
which there are no automatic atten- 
dance related withdrawals. Instead, 
students who acquire eight unexcused 
absenses are referred' to in-house 
suspension upon their return. 

The project staff was also pri- 
„ marily responsible for devel oping 
systematic discipl inary record-keep- 
ing procedures at the the two proj- 
ect middle schools. Neither school 
had adequate procedures (one kept no 
records at all). In this case, the 
national evaluation had an indi- 
rectly positive effect on implemen- 
tation. When the school with no 
record-keeping procedures resisted 
the development of such procedures, 
it was required to comply by the 
system superintendent after project 
staff informed the superintendent 
that the national evaluation 
required discipline data» 

Organized student activities 
sponsored by the SAC included not 
only the spring intramural volley- 
bal 1 program, but a dance group, a 
"Thursday afternoon discussion 
group," and a club called "Musicians 
Unl imited. " Al 1 of these groups 
were student- initiated, and the SAC 
provided space and organizational 
help. One unexpected positive bene- 
fit of the dance group is that it 
developed into a support group for 
the small number of blacks that 
attend Plymouth-Canton high school . 
The center became a place where 
bl ack students coul d meet together 
over lunch, dance , or just talk. 
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Maior Forces Affecting the 1981-82 
Program 

During the 1981-82 school year, 
two major environmental events 
caused difficulties for the Plymouth 
Project: (a) requirements of the 
national evaluation, and relations 
with the national evaluation staff 
at the Johns Hopkins University, and 
(b) a school district reorganization 
that moved ninth grade students from 
the high school to the middle 
school. In addition to these two 
major forces, the project was 
affected in less dramatic ways by 
the poor economic situation in 
Michigan, and obstacles presented by 
the physical education department in 
one of the high schools to the 
implementation of the Student Activ- 
ities Center's spring intramural 
vol 1 eybal 1 program . 

Evaluation Issues 

The relations between the 
national evaluation staff and the 
Plymouth Project remained rocky for 
much of the year (see the first 
evaluation report), but showed much 
improvement by the spring of 1982. 
Efforts in the Fall of 1981 to 
implement true experiments in the 
high school Student Services Center 
(SSC) and Student Activities Center 
(sac) met with some success, but 
problems developed because of (a) a 
drop in referrals (particularly at 
the middle school SSC) when teachers 
and administrators found that access 
to the program could not be guaran- 
teed (due to the randomization pro- 
cess ) , and (b ) randomly assigned 
referred students refusing to take 
the treatment. At the high school, 
both the SSC and the SAC did under- 
take experiments during the first 
semester, but many of the students 
that were randomly assigned to the 
treatment did not receive it. This, 
coupled with the reservation of many 
of the slots in the programs for 
continuing students and students 



referred from the Learning Options 
(LO) component, resulted in few stu- 
dents actually being experimental 
treatment and control group members. 

The result of these events at the 
middle school was that an experimen- 
tal evaluation of the middle school 
Student Services Center was not 
undertaken, and the project was 
forced to identify a non-equivalent 
control group of middle school stu- 
dents not referred to the program. 
This procedure was not successful 
(see above). 

At the high school, the project 
screened students more carefully for 
the second semester randomization 
proce ss , and devel oped a group pre- 
sentation procedure whereby poten- 
tial program participants could 
learn about the program, and could 
consent to being involved in one or 
the other program before randomiza- 
tion took place. This procedure, 
while not eliminating the problem of 
treatment refusals, decreased it 
sufficiently so that a true experi- 
ment of adequate power was carried 
out in both the high school SSC and 
SAC during the second semester of 
1981-82. 

In the me ant ime , pro j ec t st af f 
found the randomization procedure, 
the core da ta collection require- 
ments, and the administration of the 
School Action Effectiveness Study 
(SAES) questionnaire to all SSC and 
SAC treatment and control students, 
LO s t uden t s , and random sampl e s of 
the two high and middle schools put 
a severe stress on their resources 
(see the first interim report ) . A 
further difficulty was a change in 
the evaluation staff member assigned 
to the project. The Social Action 
Research Center (SARC) field worker 
assigned to the Plymouth Project 
left the evaluation in the fall, and 
the responsibility was assigned to 
the SARC Coordinator of Formative 
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Evaluation, who subsequently 

resigned in January of 1982. The 
author was assigned to work with the 
Plymouth project in mid-January 
1982. 

The project staff attributes sev- 
eral negative effects to the demands 
on their time that the evaluation 
imposed in 1981-82: (a) they had 
less time to devote to "systemic" 
goals, to working within the school 
system at large to humanize the edu- 
cational process, (b ) the project 
co-directors could not provide the 
personnel support to the SSC and SAC 
programs as they had intended, 
(c) staff coordination and planning 
decreased as staff meetings were 
cancelled to make time for evalua- 
tion activities, (d) program refer- 
rals decreased, (e) and the per- 
ceived roles of the project 
co-directors changed from that of 
direct service providers to grant 
administrators, evaluators and 
paper-pushers. The project co-di- 
rectors feared that this might nega- 
tively affect their future in the 
school system once the Federal fund- 
ing runs out.l 

It is difficult to assess how 
serious each of the foregoing 
effects has been. It is certainly 
true that there were referral diffi- 
cul ties , particul arl y at the middl e 
school, but these problems coincided 
with a re-organization that placed 
the project^'s primary source of 
referrals, ninth graders, in the 
middle schools where the project was 
new and in which referral sources 
had not yet been strongly developed. 
Concerning time for pi anning , if 
evaluation activities did interfere 



with pi anning , th is is unfortunate : 
A primary obj ective of the Program 
Development Evaluation proce ss is a 
systematic increase in planning. 

Systemic change activities, 
although an interest of the project 
(see "Policy Changes" above), have 
never been a project priority. 
Project staff have repeatedly stated 
that ^heirs is primarily a treatment 
intervention, and in fact, have 
stressed to the national evaluation 
staff their desire to* keep a low 
profile in proj ect school s. As 
regards th e ch ange in th e ro 1 e s of 
the project co-directors, part of 
, this was to be expected, because 
they are, indeed, grant administra- 
tors; the addition of a project 
staffer in charge of many evaluation 
tasks for the 1982-83 year should 
al 1 ev iate some of tho se se rv ice 
delivery vs. data gatherer distinc- 
tions. _ 

Finally, to the extent that eval- 
uation activities contributed to a 
lack of direct program support by 
the proj ect co-directors, (and this 
issue was raised by virtually every- 
one connected to the Plymouth proj- 
ect), then the national evaluation 
hurt the project. The difficulties 
derive in part from a lack of proj- 
ect staff avail abl e to pi an for and 
accomplish evaluation tasks — a 
requirement that was not clear in 
the OJJDP request for applications. 
Future requests for proposals should 
make more explicit (a) the time 
necessary for evaluation activities, 
and (b) the requirement that this 
time be budgeted for in the original 
request for support. If rigorous 
evaluation is a requirement of a 
grant, as it is in the Alternative 
Education Program, then it should be 
a maj or cr it er ion of sel ect ion for 
funding, as it was not (General 
Accounting Office, 1982), 



1 We have recently learned that the 
direct costs of maintaining the Ply- 
mouth project will be picked up dur- 
ing the 1983-84 school year by the 
local school district. 
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The difficulties the project was 
having in implementing evaluation 
requirements, the turnover in evalu- 
ation staff, plus the project's 
feeling that they were spending all 
of theii' time on evaluation activi- 
ties precipitated anger over a draft 
of the first interim evaluation 
report sent to the project for com- 
ments. The report criticized the 
pro j ect s compl iance with the 
national evaluation at a time (sub- 
mission of the third-year refunding 
request ) when the proj ect felt it 
could least afford such criticism. 
The proj ect directors therefore 
responded with a written detailed 
criticism of the report and the 
national evaluation staff responsi- 
ble for it. 

This event and its aftermath sym- 
bolize the entire year's negotia- 
tions between the project and Johns 
Hopkins: evaluation difficulties 
building to confrontation, culminat- 
ing in compromise. The first 
interim report was clarified to 
underscore that while there had been 
major problems with evaluation 
activities, the project was making 
good-faith efforts to correct them, 
and that improvements were occur- 
ring. The most important change 
that occurred was that the Plymouth 
St af f began to se e way s to ge t th e 
various tasks done (if some compro- 
mises were made), and the evaluation 
staff was willing to allow the com- 
promises and provide some help in 
order to see that the most rigorous 
evaluation possible was performed. 
For example, instead of a adminis- 
tering the SAES to a large random 
sample of the high school population 
(a task that project staff felt was 
absolutely overwhelming) , ten Eng- 
lish classes were surveyed; this was 
a compromise for both parties: the 
Engl ish classes are not as strong a 
cl imate assessment as a random sam- 



ple would be, providing a weaker 
evaluation of the school change aims 
sought by OJJDP, but the project was 
held to doing the survey, and meet- 
ing response rate benchmarks, which 
they did. Johns Hopkins facilitated 
the Plymouth survey administration 
by pre-printing mail ing label s for 
the project's parental ccasent let- 
ters. Negotiations between the par- 
ties ultimately resulted in the Ply- 
mouth project requesting funds in 
their third year budget for an eval- 
uation staff person, ' which Johns 
Hopkins strongly supported in dis- 
cussions with OJJDP staff. These 
funds were awarded for the third 
project year. 

The working relationship between 
Johns Hopkins and the Plymouth proj- 
ect is currently much improved. For 
1982-83 project personnel have 
developed a more adequate non-equi- 
val ent control group for the middle 
school SSC (collecting better pre- 
treatment data, and matching the 
non-equivalent controls more care- 
fully to the treatment population). 
They have implemented strong experi- 
ments in both high school programs, 
and have . aided in the development 
and pr e- test ing of an SAES scale 
designed to measure attituditial 
changes resulting from affective 
education. Continuing compromises 
include the quasi-experimental eval- 
uation of the middle school SSC, the 
continuing use of the English class 
sample in the high school, withdra- 
wal of the requirement to follow-up 
the 1981-82 English class sample, 
and the dropping of the Learning 
Options component from the third 
year budget (this was also due to 
the 30% budget cut requirement, but 
the lack of evaluatabil ity of Learn- 
ing Options — see the first interim 
report — contributed to the deci- 
sion) . 
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School District Re-organization 

The second major environmental 
change influencing the project was a 
district re-organization that took 
ninth graders out of the high school 
and placed them in the middle 
school. Ninth grade students were 
the primary source of referral s for 
the high school program, so the 
treatment population shifted upward 
one year to include mostly tenth 
graders. More importantly, the 
ninth grade popul ation was now in 
the middle schools in which the 
project was new and little known, so 
that referral contacts were not 
strong. It became harder to iden- 
tify the students the project feels 
are most suitable for the SSC treat- 
ment. In addition, the re»-organiza- 
tion plan diverted many of the stu- 
dents that the middle school program 
had been working with in the spring 
of 1981 to other non-project middle 
schools where they could not be 
given the follow-up support that the 
project feels is necessary to suc- 
cessful treatment. 

The re-organization also led to a 
problem for the students returning 
from Learning Options to the regular 
school curriculum. Since most of 
the Learning Options students have 
been eighth or ninth graders, it had 
been relatively easy to return them 
to the large, impersonal high school 
environment. At the middle schools 
though, the pr oj ect ran into dif f i- 
culties when principals of the two 
schools were wary of allowing the 
Learning Options participants back 
in the school. The Learning Options 
students were eventually allowed to 
re-enroll , but project staff feel 
that they are subjected to undue 
"surveillance. " 

Two additional climate effects of 
the re-organization are that both 
the ninth grade students and their 
teachers were angry about the 



change. Tlie ninth graders were 
anticipating beginning high school , 
with the autonomy it presents, and 
the ninth grade teachers felt that 
they were "demoted" from the high to 
the middle school level. 

A positive result of the re-or- 
ganization is that the SAC program 
was given more building space at the 
high school than it had expected to 
receive. It had room to conduct its 
student actvities, -and to have a 
"home" room for the ' SAC writing 
class.- 

/ 

Other Forcefield Events 

A third negative force in Ply- 
mouth's field is the severe reces- 
sionary economy in Michigan. While 
Plymouth, Michigan was not as 
directly affected as Detroit proper, 
unemployment was up, and school dis- 
trict funding, is_ largely, dependent 
upon state funding. As a whole, the 
state of Michigan is suffering a 
major fiscal crisis. 

The recession affects the project 
in two ways: First, it makes uncer- 
tain future continuation of the 
project after Federal funding is 
discontinued. This affects the 
project now, because time must be 
spent investigating and shoring up 
additional sources of funds. For 
example. Learning Options was not 
included in the third year budget, 
but is funded in 1982-83 primarily 
by local and school system funds 
(see Footnote 1). 

A second recessionary effect 
reported by project personnel at 
both the high and middle school 
level is that the students are suf- 
fering more family/home problems, 
problems directly related to unem- 
ployment . Dur ing the year , unem- 
ployment in Michigan averaged about 
12%. Project staff estimate that 
among their client population (pri- 
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marily working class) that the 
unemployment rate may be as high as 
30 to 40%. While we cannot verify 
the magnitude of either of these 
negative effects of the recession, 
they certainly seem plausible. 

As previously noted, a fourth 
negative environmental effect on the 
project was the required 30% budget 
cut for the 1982-83 year. This 
affected the project in the 1981-82 
year, because hard decisions had to 
be made about where the cuts should 
occur. Because the director of the 
Learning Options program and the 
Johns Hopkins evaluation staff had 
been unable to come to an agreement 
concerning an adequate evaluation 
design by February, 1982, the proj- 
ect decided to drop Learning Options 
from the third year funding propo- 
sal. In this manner, the SSC and 
SAC programs could be funded at 
their current level, and funds could 
be acquired for an on-site evalua- 
tion staff person. It is unfortu- 
nate that we were unable to imple- 
ment an adequate evaluation for 
Learning Options, perhaps the most 
powerful of the Plymouth interven- 
tions. 

Two forcefield events had effects 
primarily on the high school SAC 
program. In the spring of 1982, 
when the project tried to implement 
an intramural volleyball program for 
students not involved in other 
school events, the project ran into 
difficulties from the physical edu- 
cation instructors. They were wary 
of lending space and equipment to a 
venture designed for what they per- 
ceived to be marginal students. The 
SAC instructor had to see four per- 
sons, finally returning to his ini- 
tial contact before he received per- 
mission to use the gymnasium 
f acil ities after school . This 
resulted in the program starting 
later in the semester than intended. 
The project was also forced to pur- 



chase their own volleyballs, as the 
P.E. department would not let them 
use its equipment. 

A related problem that the SAC 
ran into was the lack of scheduling 
flexibility in the high school. It 
v;as difficult to schedule activi- 
ties, and most had to be held after 
school . 

Changes in the Force Field for 
1982-83 

The only anticipated major change 
in the force field is the markedly 
improved relationship with the 
national evaluation. This should 
occur for four reasons: (a) a 
highly trained evaluation staffer 
has been assigned to the Plymouth 
project as their evaluation contact 
and better continuity in and a 
smoother working relationship with 
evaluation personnel is expected, 
(b ) a ful 1-time person has been 
hired to perform evaluation tasks 
such as data collection, program 
assignment, and questionnaire admin- 
istration, (c) the procedures for 
carrying out the experiments quasi- 
experiments, and SAES questionnaire 
administration have now been "field 
tested" for a year, and (d) the 
Growth Works component, over which 
much time had been lost trying to 
develop an evaluation acceptable to 
both parties, is no longer a part of 
the federal initiative. 

Program Changes for 1982-83 

A neu intervention has been added 
in the high school Student Services 
Center to address the problem of 
school non-attendance. The data 
from the 1981-82 experiment indi- 
cated that that the SSC students 
were skipping school at a high rate. 
A group competition has been set up 
in which each "group" (class period) 
competes with the other groups on 
increasing attendance. This is an 
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?t'4'}«0>*,.» ^♦.^^jH* 11(11 tt^* 

M>'-i»*i» fit** «i4 ti, i^i^p.. 

f-m !fH»< ^t** «f lf4Hll»«Vil 

»*♦ Ifw-firti f«»f « M»f«ti4fl 
^Mt) Wfer* niftiti^ M«- 

*»«|tn H*rf f#ii if»»t iiti« 
tl«»4 fii mi ^mta Urn 
iHiip m?t-'*-^***^i III t%««, 

*i4<Kibtwi {1*4.4 fwf liti-ii. 

ti^« iii»it^ iKi Hie nc 

fl^M* rfi««««f mi f«l«iiwi, • 

ii«€4»4Nlii«l Wt lllVM UiMM>i «i|t U 

lt<»#r (««t«« miliitili will W 

«!«## Mti«}tl«« #m te> H«M»»4* m4 

r#MUi k»«r Ctli* 

«t«MM4««»« ptM*#4 yftfirMfco*^, mi 

mU4U¥0 wf^ic*. KWm will k» 
AN>MP»i t« MfiMittr* mttf «r tli» 



«J«fitife which th«y often get Involved 

A further change involveii th« uii« 
»f iMtur*. Sn l¥fl)-82| non-proRrflm 
kenierft war* u«fd «i tutors, but the 
iAC ^«»H'u«tor fplt that ihli proco- 
dur# v«ii not Mtiaf actuary « bec4uue 
Ihv SAC fttud^ntt rt«tf<nir<l thi< oxper- 
l4*# t>t th« iutMta, In 1982-83 auc- 
e«ft*ful graduatca of ths prosram 
vill da tutoring. 

yiR^lty. U\« gAC inatructor vaa 
the wniof elaaa adviaor during the 
I9ftj-tt2 y^»aT, and thrlr roproaenta- 
, tiv« group operated out of hia 
office. Tor the following year, he 
will b« thw, tenth grade adviaor 
where he feel a he can be more of a 
counieling aid. The SAC inatructor 
will alao advite the Student Forum 
(tb« high achool atudent govern- 
•enth Md that body will operate 
•tit of the gAC officea. 

At the «iddle achool SSC, leaa 
lia« win be apent on the one-to-one 
bondititi md the groupa will be 
itarted earlier in the aemeater. 
The prograa'a atudent evaluation 
inatruBient, thtt Student Behavior 
tvaluacion (Sfii) h«s been aiaplified 
for 1982-BS. the inatrtnent waa 
^eaigned for uae at the high achool 
level, Md the middle achool ataff 
feel that it ia too detailed and 
ceaiplicated for middle achool uae. 
Data froa an evaluation of the 5BE 
aupporta their aaaeaaaent.^ 



»«»aM^-i>>«»4M»«. » » » ik.^ » » «^ a» ^ UK: ■» 

S 8<M ieport on utility for dlagno- 
ais of atudent Behavioral Evaluation 
tcaUa, April. 1982. Available froa 
author. 
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Table 1 

Outcomes on Measures of Plymouth High School 
Student Services Center Goals 



Goals 



Treatment 



M 



SD 



Control 



M 



SD 



Reduce alienation 

Alienation 

Improve academic skills 

Self-reported school 
effort 

Self-reported grades 

8182 G.V.A. ^ 

Classes attempted 

Classes passed 

Percentage of classes 
passed 

Reduce delinquent behavior 

Self-reported delinquency 

Self -reported serious 
delinquency 

Drug use 

Truancy petitions 

Improve coping behaviors' ^ 

Interpersonal competency 

Rebellious autonomy 

Practical knowledge 

Increase parental involve- 
ment in education 

Parental emphasis on 
education 



• 44 



• 33 



,27 50 



.29 52 



.43 



.30 



• 25 42 



• 27 42 



.88 



.45 



1.79 


.76 


43 


1.91 


.65 


43 


.41 


1 • 00 


1 • J J 




1 • OZ 


1 ^ "7 




. d2 


5.34 


.77 


44 


4.67 


1.30 


45 


. 004''- 


4.24- 


2.06 


46 


3.91 


1.99 


46 


.44 


.77 


r35 


42 


.76 


,31 


44 


.93 


.42 


.20 


46 


.37 


.21 


38 


.27 


.22 


.23 


46 


.20 


.23 


39 


.63 


.89 


.17 


42 


.78 


.30 


38 


.05* 


0.00 


0.00 


57 


.02 


.13 


60 


.33 


.73 


.30 


50 


.84 


.20 


41 


.05* 


.82 


.27 


51 


.72 


.35 


42 


.11 


1.35 


.40 


51 


1.34 


.49 


42 


.91 


.32 


.28 


46 


.42 


.49 


42 


.007* 



Statistically significants 

1 The Expression Scale was completed by too few control students for analysis • 

2 A very weak measure of achievements 
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Table 2 

Outcomes on Measures of Plymouth High School 
Student Service Center Objectives 



Treatment 



Objectives 



Control 



M 



SD 



M 



SD 



Increase attendance 

SAES non-attendatice index 1.52 
Classes missed second 102.86 

semester 
Classes missed per 

class attempted 

Reduce official voluntary 
withdrawals 
Voluntary withdrawals 
Voluntary and invol- 
untary withdrawals 

Increase sense of belong- 
ing and Purpose 
Attachment to parents 
Attachment to school 
Alienation 
Involvement 

Decrease expulsions 
Expulsions 

Administrative with- 
drawals 

Increase self-esteem 
Self-esteem 

Increase participants ' 
ability to own respon- 
sibility for their own 
behavior 
No measure 

Reduce out-of-school 

suspensions / 
Out-of-school suspensions 

Increase parental involve - 
ment in schools 

Parental emphasis on See Table 1. 

education 

Develop more positive par- 
ental attitudes toward 
school and alternative 
education 
No measure 

Other theoretically rele- 
vant measures 
Belief 

Internal control 
Self-reported suspensions 
In-school suspensions 



.57 52 
45.70 43 



1.64 
73.89 



.57 44 
46.74 45 



.32 

.004** 



19.80 


9.22 


44 


15.95 


10.21 


43 


07* 


9%' 




57 . 


21% 


— 


61 


.10* 


11% 




57 


21% 




61 


.18 
















.41 


.24 


52 


.41 


.29 


44 


.98 


.46 


.25 


50 


.47 


.27 


43 


.86 


See Table 1, 










.07 


.10 


52 


.13 


.13 


42 


.02** 


0.00 


0.00 


54 


0 .'oo~ 


"b.oo" 


~00~ 


"1.00 


.56 


1.31 


55 


.70 


1.74 


54 


.64 


.58 


.18 


49 


.62 


.18 


41 


.35 


.08 


.27 


.53 


.29 


.89 


52 


.10* 



.64 


.25 


51 


.66 


.24 


42 


.74 


.48 


.29 


51 


.49 


.27 


42 


.88 


.31 


.47 


51 


.28 


.45 


43 


.72 


.34 


1.00 


53 


.25 


.58 


53 


.55 



* strong trend. 

** Statistically significant. 



Table 3 

Plymouth High School Student Services Center Has Negative 
Effects on First Semester 1981-82 Students 





Treatment 






Control 




p 


Goals 


M 


SD 


N 


M 


SD 


N 




Interpersonal Competency 


.76 


.27 


21 


.91 


.14 


17 


.05* 


Self-esteem 


.52 


.19 


20 


.73 


.12 


17 


.ooo; 


Rebellious autonomy 


.92 


.14 


21 


.73 


.36 . 


17 


.03* 


Involvement 


.06 


.06 


23 


.12 


.12 


17 


.05* 


Attachment to school 


.41 


.27 


22 


.62 


.25 


17 


.02* 


Parental emphasis on 
education 


.31 


.31 


21 


.54 


.32 


17 


.03* 


Invalidity^ 


.10 


.17 


21 


.11 


.14 


17 


.98 



A measure of truthfulness in reporting. 
*Statistically significant. 
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Table k 

Parental Contact Data, Plymouth 
Second Semester, 1981-82 



Parent Phone Calls Parent Visits Parent Meetings Total Parental Contacts 

M SD M SD M ■ SD M SD 



High School SSC 


2.26 


2.59 


M 


.72 


.87 


1.108 


3.61 


3.94 


Middle School SSC 


5.32 


4.11 


.81 


1.16 


■ .41 


.99 


6.63 


5.15 


High School SAC 


1.76 


1.72 


.19 


.50 


.33 


.12 


2.28 


2.27 


Growth Works 


9.68 


7.91 


4.46 


5.52 


.89 


1.15 


15.03 


11.41 
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Table 5 

Pretest Measures on Plymouth Program 
Participants and Controls 





High School 


ssc 


High School 


SAC 


Middle 


School 


SSC 


Growth Work; 








£ 


T 


C 


2. 


T 


C 


£ 


T 


Attachment to parents 


6.6 


5.8 


.14 


6.8 


6.7 


.92 


6.6 


6.8 


.49 


6.7 


Affection from parents 


6.0 


6.2 


.77 


5.9 


6.2 


.69 


6.1 


6.2 


.81 


6.5 


Alienation 


1.7 


1.9 


.55 


1.7 


1.6 


.94 


2.0 


1.4 


. 001** 


1.4 


Attachment to school 


4.7 


4.0 


.26 


5.9 


5.04 


.46 


3.03 


4.4 


.01** 


5.0 


Belief 


4.1 


4.0 


.95 


3.6 


2.8 


.17 


2.7 


2.3 


.07* 


2.8 


Expression 


2.0 


1.9 


.92 


2.7 


3.6 


.12 


2.2 


2.8 


. 04** 


2.9 


Interpersonal compe- 
tency 


3.3 


3.3 


.91 


2.8 


2.5 


.61 


2.2 


1.7 


..01** 


1.8 


Self-esteem 


7.5 


6.5 


.07* 


6.6 


5.0 


.03** 


4.9 


5.0 


.72 


5.2 


Internal control 


2.9 


2.7 


.63 


2.' 


2.6 


.63 


2.2 


2.2 1 


.00 


2.5 



* strong trend. 

Statistically significant. 
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Table 6 



Correlations Between Time in SSC Class 
and Selected Client Characteristics: 
Plymouth High School SSC Class 



Time in SSC Class 
Client characteristic r p 

Parental meetings -.11 .47 

Parent phone calls -.13 .37 

Parent visits -.09 .53 
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Table 7 

Percentage Completing Student Questionnaire 
Plymouth, Spring, 1981-82 





Treatment 


Control 


High School SSC 


81% 


71% - 


Middle School SSC 


91% 


79% 


High School SAC 


81% 


80% 


Growth Works 


62% 




High School English Classes 


97% 




Middle School Random Sample 


99% 
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Table 8 

Results oil Measures of Plymouth High School 
Student Activities Center Goals 



Treatment 



Control 



Goals 



Reduce alienation 

Alienation 
Improve academic skills 

Capitalization^ 

Punctuation^ 
1 



Words 
Sentences 



1 



Sentence structure 
Study skills^ 
Essay ' 

Self-reported grades 

. 8182 G.P.A. 

Self-reported school 
effort 

Classes attempted 

Classes passed 

Percentage of classes 
passed 

Reduce delinquent behaviors 

Self-reported delinquency 

Self-reported serious 
delinquency 

Drug use 

Truancy petitions 

3 

Improve coping behaviors 

Interpersonal competency 

Rebellious autonomy 

Practical knowledge 

Increase parental involve- 
ment i^ . education 

Parental emphasis on ed. 



M 


SD 


N 


M 


SD 


N 


P 


• JO 






.JO 


. Jl 


± / 






1 on 


jt 


i . Uo 


J .3 / 


13 


. 77 


4.91 


.. 3.15 


34 


-.38 


3.15 


13 


.0000** 


4.29 


?.54 


34 


.54 


■ 3.27 


13 


.002** 


3.74 


4.70 


34 


1.15 


4.86 


13 


.10* 


2.09 


5.22 


34 


1.62 


5.33 


13 


.78 


5.29 


4.19 


34 


.62 


4.72 


13 


.002** 


7.06 


11.41 


36 


-12.50 


10.62 


6 


.0004** 


2.08 


.55 


36 


_ 1..61_ 


_ .78_ 


_18_ 


_..01^* _ 


1.62 


.89 


34 


1.52 


.63 


21 


.66 


.29 


.28 


36 


.29 


.21 


17 


.94 


5.41 


.82 


34 


4.73 


1.28 


22 


.02** 


A T7 




j4 


3 . d4 


1.43 


22 


.13 


.80 


.27 




.65 


.43 


22 


.12 


.28 


.20 


32 


.34 


.34 


14 


.35 


.14 


.19 


32 


.21 


.22 


14 


.28 


.59 


.39 


34 


.63 


.35 


14 


.77 


0.00 


0.00 


34 


0.00 


0.00 


25 


1.00 


.81 


.21 


33 


.75 


.27 


17 


.44 


.85 


.24 


33 


.88 


.27 


16 


.72 


1.48 


.47 


32 


1.13 


.64 


17 


.03** 


.32 


.29 


34 


.47 


.30 


• 17 


.10* 



ERIC 



1 This is the average change from pre- to post-test. 

2 Very low n*s make this analysis inconclusive* 

3 The Expression scale was not completed by any control students. 
* Strong trend. 

** Statistically significant. -464- 351 



Table 9 

Results of Measures of Plymouth High School 
Student Activities Center Objectives 



ERLC 



Objectives 



M 



Treatment 
SD 



N 



Increase attendance 



class attempted 

Decrease official vol- 
untary withdrawals 

Voluntary withdrawals 
Voluntary and involun- 
tary withdrawals 

Increase sense of belong- 
ing and purpose 

Attachment to parents 
Attachment to school 
Alienation 
Involvement 

Reduce expulsions 

Expulsions 0 
Administrative with- 
drawals 

Increase self-esteem 

Self-esteem 

Increase ability to ov/n 
responsibility 

No measure 

Reduce out-of-school 
suspensions 

Out-of school suspensions 

Increase parental involve- 
ment in schools 

No measure 

, Develop more positive parental 
attitude toward school and 
alternative education 

No measure 

Other theoretically relevant 
measures 

Belief 

Internal control 

Self-reported suspensions 

In-school suspensions 

^Statistically significant 



.11 



.49 44 



K 



Control 
SD~ 



SAES non-attendance index 


1 


.53 


.61 


36 


1 


.63 


.50 


Classes missed second 


91 


.41 


55.79 


34 


79 


.82 


50.69 


semester 
















Classes missed per 


17 


.05 


10.14 


34 


18 


.55 


13.71 




16% 




44 


8% 




25 


.57 


16% 




44 


8% 




25 


.57 


.45 


.27 


36 


.32 


.26 


18 


.12 


.49 


.30 


35 


.40 


.31 


17 


,36 


.36 


.23 


32 


.56 


.31 


17 


.02* 


.06 


.09 


34 


.03 


.08 


16 


.25 


.00 


0.00 


44 


0.00 


0.00 


25 


1.00 


.52 


1.35 


44 


.76 


1.33 


25 


.48 


.64 


.17 


30 


.59 


.24 


13 


.45 



.08 



,28 25 



.75 



.59 


.24 


33 


.54 


.27 


17 


.48 


.58 


.23 


33 


.56 


.28 


17 


.81 


.23 


.43 


35 


.29 


.47 


17 


.62 


.20 


.55 


44 


.44 


1.08 


25 


.24 
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Table 10 



Correlations of T\^o Implementation Measures 
(Parent Contacts and Time in SAC Class) 
with Selected Client Characteristics 
for the Plymouth Middle School 
SAC Random Treatment 



Implementation measure and 

client characteristic r 



T otal parent contacts 

Administrative removals .40 .003 

Out-of-school suspensions .48 .001 

Total suspensions .43 .004 

Classes missed .60 .001 

Classes passed -.32 - - - .06 



81-82 G.P.A. 



.32 



.06 



Expression Pretest 



.71 



.004 



Time in SAC class 



Parental meetings 
Parent phone calls 



.09 



.00 



.98 



.58 



Parent visits 



.03 



.85 
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Table 11 

Outcomes on Measures of Plymouth Middle School 
Student Services Center Goals 



Goals 



Treatment 



M 



SD 



Non-equivalent Control 
M SD N 



Reduce alienation 

Alienation ,46 ,25 38 ,50 ,25 76 

Increase academic skills 

Self-reported grades 1.53 ,94 36 ' 1,93 ,77 

Self-reported school ,15' ,18 35 ,37 ,32 

effort 



81 
83 



.57 

,02^^^ 
,00002* 



Classes attempted 


7.32 


1.13 


43 


7.37 


1.17 


98 


.83 


Classes passed 


5.72 


2.19 


43 


6.65 


1.49 


98 


. 004''^ 


Percentage of classes 


.76 


.25 


43 


.91 


.14 


98 


.0000* 



passed 



Self-reported delin- 
quency 


.60 


.29 


32 


.40 


.27 


74 


.0009* 


Self-reported serious 
delinquency 


.53 


.31 


32 


.33 


.28 


75 


.002* 


Drug use 


.80 


.30 


49 


.52 


.39 


77 


.0000* 


Truancy petitions 


.02 


.15 


45 


0.00 


0.00 


105 


.13 


Detentions 


1.60 


2il7 


20 


1.31 


1.71 


52 


.55 


Improve coping behaviors 
















Interpersonal competency 


.63 


.24 


36 


.70 


.24 


75 


.18 


Rebellious autonomy 


.65 


.31 


35 


.70 


.34 


76 


.43 


Practical knowledge 


1.33 


.37 


34 


1.33 


.51 


78 


.97 



Increase parental involve- 
ment in education 

Parental emphasis on 



,50 



.29 36 



.A9 



,27 80 



.87 



*Statistically significant. 



ERIC 
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<.M^*<Hi»* m Himmftsii «»f Plyvmich Middle School 



ERIC 



'twill $ JMRll i^lN^WjlW^"" 



Mon-tfquivalenc Control 



SD 



N 



til 

tit 



,7J 
11.07 



37 
*3 
43 



45 

45 



l.ll 
5.02 
5.36 



6X 
6X 



.88 84 
.70 99 
5.60 99 



109 
109 



? "fUT?"' S.~T>J" J.. ■ • I^CtJHUJs?? FBI' Till US? TI Wt.V 



.a 



.16 12 



.58 



.15 75 



1.9) 3.29 U 



.66 



1.23 106 




llMii'MHiiiiiMi 



.01** 

.61 

.03** 



.51 
.51 





.1? 


.37 


37 


.42 


.29 


85 


.41 




.21 


.31 


36 


.36 


.27 


82 


.09* 




.»l 


.34 


36 


.46 


.24 


79 


.25 




.M 


.14 


36 


.15 


.19 


80 


.10* 




















0,00 


0.09 


45 


0.00 


0.00 


106 


1.00 




1.30 


6.09 


44 


2.00 


4.40 


102 


.000: 



.004** 



.0000** 



*6i 


,U 


35 


.55 


.25 


78 


.10* 




.36 


n 


.46 


.27 


78 


.22 




.4S 


35 


.49 


.50 


80 


.09* 




1.21 


30 


i.oa 


1.34 


52 


.28 






H . . . 






74 


,08* 
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Table 13 



Correlations of Two Implementation Measures 
(Parent Contacts and Time in SSC Class) 
with Selected Client Characteristics 
for the Plymouth Middle School 
SSC Treatment 



Implementation measure and 
client characteristic 



Total parent contacts 

Excused absences .35 .02 

Administrative removals .26 .09 

Truancy petitions .47 .002 

Court referrals .44 .004 

Time in SSC class 

Parental meetings .11 .69 

Parent phone calls .09 .73 

Parent visits -.05 .84 
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Table 13 

Correlations of Two Implemei 
(Parent Contacts and Time 
with Selected Client Chi 
for the Plymouth Midc 
SSC Treatment 



Implementation measure and 
client characteristic 

Total parent contacts 
Excused absences 
Administrative removals 
Truancy petitions 
Court referrals 

Time in SSC class 



Educational Improvement Center — South Alternative Education Project: Second 

Interim Report 



D. E. Rickert, Jr. 



The Education Improvement Center- 
South (ElC-South) is a technical 
assistance and resource center. Tlie 
primary clients are educators, but 
many community groups and individu- 
als use the resources of the Center. 
The catchment area for the Center is 
southern New Jersey, which is com- 
prised of Atlantic, Camden, Cape 
Kay, Cumberland, Gloucester, and 
Salem Counties. 

The Alternative Education Project 
is one of many programs sponsored by 
ElC-South. It is the only project 
within the larger agency that pro- 
vides direct services on a large 
scale. The details of ElC-South and 
its. organizational arrangements are 
described more fully elsewhere (Ric- 
kert, 1982). 

During the 1981-82 school year, 
the Alternative Education Project's 
delinquency prevention efforts were 
targeted at four schools: Pleasant- 
ville Middle School in Pleasantville 
and Cleary Middle School in Buena, 
both in Atlantic County, D'ippolito 
Elementary School in Vineland, Cum- 
berland County, and Pennsauken Jun- 
ior High in Pennsauken, Camden 
County. 

Probl ems » Goals and Guiding Princi - 
ples 

During the 1981-82 school year 
the explicitly identified problems 
and goals (i.e., those identified in 
the Program Development Evaluation 
plan) were the same as described in 
the first Interim Report , but dis- 
cussion with program staff made it 
apparent that the proj ect does, not 
operate on a "theory" in the sense 
that it is used in Program Develop- 
ment Evaluation (PDE; Gottfredson, 



Rickert , Gottfredson & Advani , 
1983). Rather, the project is based 
on a flexible set of ideas or 
hypotheses. In the ElC-South Alter- 
native Education Proj ect , staff do 
what seems most appropriate at the 
time . 

The basic guiding ideas of EIC- 
South's Alternative Education Pro- 
ject are best presented in the pro- 
ject staff's own v/ords ; they appear 
as an appendix to this report (see 
Appendix A) . 

Objectives 

In the first Interim Report many 
obj ectives and suggested measures 
were described. Measurement was 
only implemented ^f ol: ^ some ~of these 
(see Table 1). Project staff mem- 
bers were uncomfortable with the 
structure elicited using the PDE 
model, and felt a more realistic 
description of their project was 
preferable. At the end of the year, 
th refore, project administrators 
revised the explicit statement of 
the problems their program was 
addressing and the corresponding 
goals and objectives (see Table 2). 

Interventions During the 1981-82 
School Year 

In the first Interim Report s five 
general components of the ElC-South 
program were described: (a) a 
school climate improvement compo- 
nent, ( b ) a communi ty pr oce s s compo- 
nent, (c) a youth participation com- 
ponent , (d ) a publ ic relations 
components and (e) a leadership and 
training resources (technical assis- 
tance ) component . 
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In reality, the major thrust of 
the program, in terms of direct ser- 
vices provided, were alternative 
education classes, called School/ 
Community Relations classes, held in 
each of the four target districts; a 
"self-contained" class in Penn- 
sauken;^ and "Community-Process" in 
two of the target districts. Vine- 
land and Pennsauken. Many of the 
target students also received indi- 
vidual counseling from ElC-South 
personnel. A small number of stu- 
dents participated in "special 
clubs" (e.g. dance club, etc). 
Some tutored students in lower 
grades and some received tutorial 
'services. (See Tables 3 and 4.) A 
student referred to the Alternative 
Education Project could receive as 
few as one treatment or a combina- 
tion of as many as six of the listed 
treatments. This depended upon what 
project personnel perceived the stu- 
dent's needs to be, as well as on 
scheduling convenience. The follov;- 
ing sections describe the project's 
major interventions. 

School /Community Relations Classes 

The way in which this interven- 
tion v/as executed evolved during the 
1981-82 school year. Tliere was not, 
at the outset, a clearly articulated 
set of implementation blueprints, 
although training in preferred 
approaches took place during the 
preceding spring and summer. The 
fact that there were no blueprints 
for implementation is related to an 
incremental approach to planning 
followed by this project. The 



1. The "Self-Contained" class in 
Pennsauken existed pr ior to the 
ElC-South Alternative Education Pro- 
gram as a special class for students 
with behavior problems. When the 
ElC-South program became opera- 
tional, the self-contained class was 
adopted as part of the program. 

-47 



proj ect director explained that it 
was important for Teacher/Counselors 
(those responsible for carrying out 
this intervention) to gain experi- 
ence in their assigned schools and 
then execute the program as seemed 
appropriate. 

In short, the Teacher/Counselors' 
activities are not guided by a sin- 
gl e model , but rather by a casework 
approach where vjhat seems intui- 
tively correct becomes the interven- 
tion. 

The general aims of the School/ 
Community Relations Classes were the 
following : (a ) teach students com- 
'^munication skills, (b) help students 
improve their self-image, (c) work 
at resolving students' family prob- 
1 ems , ( d ) improve study skil 1 s , and 
(e) aid students in career explora- 
tions. 

One Teacher/Counselor was 
assigned to each of the four target 
schools. Students chosen to parti- 
cipate met several times a week with 
their Teacher/ Counselor in a class- 
room setting. Teacher/Counselors 
seemed to have a lot of flexibility 
to do whatever they thought best 
with particular students, subject to 
guidance by the Educational Direc- 
tor . They were responsible for 
keeping the Educational Director and 
Project Director informed of their 
treatment plans, attendance, paren- 
tal contacts, and schedules. Ser- 
vices provided included counseling, 
home visits , tutoring in basic 
skills, and arranged meetings with 
community leaders. 

In addition to class time with 
students , Teacher/ Counselors devoted 
a considerable amount of time to 
providing advice and consultation to 
other teachers in the school, parti- 
cularly teachers of participant stu- 
dents. They reportedly also met 
frequently with vice-principals with 
regard to discipline policy. 
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Coininents by principals (1982 
Principal Questionnaire) at the four 
target schools indicate that the 
project v;as perceived as a program 
to improve students' attendance, 
grades, discipline, and self-image 
through individual counseling and 
basic academic instruction. One 
principal of a project target school 
stated that he woul d 1 ike the pro- 
ject to develop more of a blueprint 
for operation such as a curriculum 
guide. The 1982-83 program is based 
on a more explicit curriculum guide. 

Hoy students yere selected for 
the School/ Community Relations 
Classes . As early as the proposal- 
writing stage, nine selection crite- 
ria were agreed upon by Alternative 
Education Project staff. They are 
the following: (a) poor attendance 
record, (b) high number of suspen- 
sions, (c) low academic performance, 

(d) perceived "drop-out potential," 

(e) history of delinquency, 

(f) history of discipline problems 
in school, (g) court involvement for 
status offenses, and (h) high degree 
of parental commitment to partici- 
pate. Also listed as a criterion 
was that students should be referred 
by school administrators, teachers, 
school counselors, or parents. 
Early discussions with the national 
evaluation staff led to a plan to 
develop pools of at least 60 stu- 
dents meeting proj ect criteria at 
each school; 30 were to have been 
randomly assigned to the treatment, 
and 30 to the control group. 

At the end of the 1980-81 school 
year, copies of the criteria for 
student selection were distributed 
to all administrators, guidance 
counselors, teachers, and principals 
who had contact with sixth grade 
students. Meetings with the the 
potential referring agents were held 
in each district to determine refer- 
rals for the project. Each team of 
proj ect staff , consisting of a 
Teacher/Counselor and a Community 



Process Consultant, then conducted 
student-family interviews to deter- 
mine which youths would volunteer 
for the project. Teacher/Counselors 
then met with school guidance 
counselors to schedule volunteers' 
project participation into their 
school schedules. 

The project did not receive 
enough referrals to constitute 
treatment and control groups ais 
planned. 

Self-Contained Class in Pennsauken 

At Pennsauken Junior High School, 
the ElC-South Alternative Education 
'Project also operates a Self-Con- 
tained Class. Students assigned to 
this class stay in the same class- 
roa.a for the entire day. The Penn- 
sauken Self -Contained Class was in 
operation several years prior to the 
ElC-South Alternative Education Pro- 
ject and was ^incorporated "into* the 
Project when it began. 

An alternative-Education Project 
staff member is assigned full time 
to the self-contained class. He 
conducts the School /Community Rela- 
tions curriculum daily (i.e., commu- 
nication skills , self-image , family 
probems , study skills , and career 
exploration), teaches history, and 
supervises the students' study per- 
iod. In addition to the ElC-staff 
member, three staff teachers, spe- 
cially selected for their ability to 
deal with "problem" students, visit 
each day to give instruction in 
mathematics, science, and English. 

The same criteria as for the 
School /Community Relations classes 
were used to select students for the 
Self -Contained Class except that 
certain students yere judged to be 
more likely to benefit from the 
Self -Contained Class. The criteria 
underlying these judgments are not 
specified. 
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Community Process Component 

Three Community Process Consul- 
tants were employed by ElC-South 
during the 1981-82 school year. One 
worker was assigned to Vineland, one 
to Pennsauken, and one to both Buena 
and Pleasantville, since they are in 
the same county. Apparently, only 
in Vineland and Pennsauken was the 
component fully implemented to the 
extent that students actually parti- 
cipated (see Table 3). 

The Community Process Consultants 
were charged with primary responsi- 
bil ity for coordinating in-service 
training for all teachers in the 
target districts, covering such top- 
ics as violence and vandalism, dis- 
pute resolution, parenting, teacher 
effectiveness , counsel ing services , 
the criminal justice system and dis- 
cipline. Another major function of 
the Community Process Consultants 
was to call the homes of project 
target students who were absent. 
They also provided counseling ser- 
vices for a small number of stu- 
dents. 

The Community Process Consultants 
had key responsibil ity for organiz- 
ing large committees of community 
figures and school personnel. Some 
students served on these committees 
(See Table 3). The purpose of these 
meetings was to attempt to put into 
action the Project Director's theory 
about how institutional change 
occurs (See Appendix A). 

Generally, large meetings were 
called where key community members 
would discuss community problems and 
their causes. This process, in and 
of itself , was considered to be 
therapeutic . A hoped for outcome 
was that peopl e with power in the 
communities would become more aware 
of the problems that youth experi- 
ence and then do something about 
them. The actual outcomes of these 
meetings are, by and large, unknown. 



although the project director per- 
ceives that awareness was increased. 

Project Process 

V7e do not know as much about the 
implementation of this project as we 
woul d 1 ike . The ad hoc na tur e of 
program operations makes it diffi- 
cult to monitor implementation. Our 
early attempts to get project manag- 
ers to be more explicit about what 
they planned tc Jo and why may have 
served to alienate them. Much of 
the early numeric feedback relayed 
to proj ect personnel was seen as 
abstract and, therefore us el ess. 
Put another way, both the spontane- 
ous nature of the project and dif- 
ferences in the language -^nd per- 
spectives of implement ers and 
evaluators have sometimes made it 
difficult for us to engage in useful 
proce ss evaluation. 

Interviews with key project staff 
indicate that the project experi- 
enced no substantial opposition dur- 
i its first year of operation. 

^.afl I ^ rt that key school system 
personnel have become more receptive 
to the project over time. * This 
accords with the comments made by 
principals in the 1981-82 Principal 
Questionnaires. But by and large, 
the EIC- louth Alternative Education 
Project is a "black box." At the 
end o the 1981-82 program year it 
was evident that the PDE plainnive 
^ jcess was not being used by pv^: 
ject personnel in the spirit tha^. it 
was intended, and therefore was roi: 
providing useful information ab;^t.:c 
project plans and implementation. 

Fortunately,, there is rv js? a to 
believe that by the end of the 
1 982-83 school year, the pi o j ect 
will be less of a black box be cause 
the project developed a ' new plan 
according to its own model, and 
motivation to document implementa- 
tion is now higher. A clear 
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workplan with milestones has been 
worked out. Also, the project has 
reworked its theory into a guide for 
action that personnel see as mean- 
ingful, and reduced its list of 
goals and objectives. Finally, a 
full-time Evaluation Coordinator was 
hired by the project. 

Interim Outcome Evaluation 

Design Evolution 

After earlier plans to conduct a 
true experiment were abandoned (see 
the first Interim Report ) , new plans 
were made in August, 1981 for a qiia- 
si-exper iment . The new design 
called for identification of non-e- 
quival ent control groups in each of 
the target schools. Also, at the 
suggestion of the Project Director, 
control schools were to be chosen 
for each of the four target schools, 
matched on demographic characteris- 
tics (i.e., SES, Race, school size, 
average achievement test scores and 
school grade structure). A college 
student was hired part-time to do 
the work involved with collecting 
the data for matching schools. 
Within each control school referrals 
were to have been solicited for sev- 
enth and eighth graders based on the 
same nine criteria as in the target 
schools . 

Because of the non-equivalent 
nature of comparison groups the 
design depended on the administra- 
tion of a pre-test survey in the 
fall of '81 to all treatment and 
control students, the controls in 
the target schools as v;ell as in the 
control schools (see Table 5). 

By late October, 1981, the pro- 
ject was not ready to administer a 
pre-test. With the assistance of 



the OJJDP project officer and a mem- 
ber of the Polaris (Technical Assis- 
tance) staff, a meeting was held at 
the OJJDP office in Washington to 
discuss the importance of the 
e^^aluation and obstacles to proceed- 
ing. The project director did not 
leel that the evaluation would be 
useful and had reservations about 
the appropriateness of collecting 
information about individual s in a 
government- sponsored demonstration 
project. The meeting did not result 
in an immediate solution, but in 
January, 1982, a full time Evalua- 
tion Coordinator was hired by the 
project. In the weeks following the 
meeting and during a site visit made 
'to ElC-^outh in January 1982, new 
arrangements for evaluating the pro- 
ject were made. The idea of "con- 
trol*' schools was abandoned as 
unfeasible. Again, the agreed upon 
design was a non-equivalent control 
group comparison. Students who were 
in trea"tment as of January~~30', '1982 
were to comprise the treatment 
group. All other students on the 
vaiting list and new referrals would 
be in the comparison group. Tliis 
design, of course, depended on the 
program having reached a stable 
stage whereby new referrals and 
waiting list students would not 
enter treatment during the year. In 
lieu of a survey pre-test, the pro- 
ject was to collect pre- treatment 
data from records from the 1980-81 
school year for all treatment and 
control students . 

The expectation that treatment 
students would remain in treatment 
and control students would remain 
untreated was not met. By March, 
1982, virtually all students in the 
control group had received some type 
of service (see Table 6). 
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Relationship Between Kno\^m Back - 
ground Characteristics of Target 
Students and Amount of Various 
Treatments Received 

Because Fall , 1981 , pre-tests 
were not administered as initially 
planned, there v/ere few pre-treat- 
ment data avail abl e for a thorough 
exploration of the relationship bet- 
ween background characteristics and 
treatment. 

Several individual items and sca- 
les from the 1982 Spring Student 
Questionnaire can be viewed as mea- 
sures of background characteristics, 
for instance, the Parental Education 
scale. Correlations between this 
variable and the amount of treatment 
received were non-significant.. The 
correlation between student self-re- 
ported sex (coded 0=female, l=male) 
and the number of quarters in a spe- 
cial club was significant (r=-.21; 
p<.05). The correlations between 
students' self-reported age and 
number of quarters in community pro- 
cess (r=.22; p<.01), counseling 
(r=.18; p<.05), and self-contained 
class (r=.48; p<.001) were signifi- 
cant, indicating that the older tar- 
get students tended to receive these 
treatments . Self-reported grade 
level was significantly positively 
correlated with the number of quar- 
ters in the self-contained class 
(r=.40; p<.001) and tutoring (r=.19; 
p<.05) . Self-reported race (coded 
l=white , 0=all others ) was corre- 
lated significantly with the number 
of quarters in tutoring (r=-.22; 
p<.05) , school /community relations 
class (r=.19; p<.05) , community pro- 
cess (r=.47 ; p<.001) , counseling 
(r=.28; p<.01), and self-contained 
class (r=.35; p<.001). The implica- 
tion is that the students partici- 
pating 1 onge St in school / community 
relations class counseling and the 
self-contained cl ass tended to be 
white , whereas students receiving 
tutoring tended to be non-white. 



Several additional measures of 
background characteristics were 
available from school records for 
the previous (1980-81) school year. 
These were: number of absences dur- 
ing the 1980-81 school year and num- 
ber of grade retentions in the stu- 
dent's history. Data on 1980-81 
suspensions were provided but were 
missing for most students . Indivi- 
dual subject grades for the 1980-81 
school year were provided but were 
not used because , for two school s , 
many of the correlations between 
individual 1980-81 grades were 
either peculiarly low or negative 
(see Table 7) and correlations bet- 
ween 1980-81 and 1981-82 overall 
CPA's were low for Vineland (r = 
.02) and negative for Pennsauken (r 
= -.19). This pattern of correla- 
tions suggests that the data may 
have been incorrectly transcribed. 

Number of ab sence s dur ing th e 
1980-81 school year was signifi- 
cantly positively correlated with 
community process participation 
(r=.18; p<.05), counsel ing (r=. 16; 
p<.05), participation in special 
clubs (r=.23; p<.01), and participa- 
tion in the self-contained class 
(r=.28; p<.01). 

The number of retentions in stu- 
dents' histories was significantly 
positively correlated with community 
process participation (r=.24; 
p<.01) , participation in special 
clubs (r=.18; p<.05), and participa- 
tion in the sel f -contained class 
(r=.34; p<.001). These correlations 
are summarized in Table 8. 

Implications of these correla - 
tions . The correlations just 
described imply that students parti- 
cipating in the various program com- 
ponents are different in important 
ways from students who did not par- 
ticipate — i.e., the comparison group 
is indeed non -equivalent. It will 
therefore be necessary to attempt to 
adjust statistically for these 
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pre-existing differences in search- 
ing for the effects of participation 
in the various project coraponents. 

Analytic Methods 

Because, by the end of the 
1981-82 school year, there were vir- 
tually no control students , a 
regression approach to data analysis 
was applied, using number of quar- 
ters in various treatments as pred- 
ictors in separate regression equa- 
tions . All target students in 
schools v;here treatments were imple- 
mented were included in each analy- 
sis, but if a treatment was riot 
implemented in a school, that school 
was not included in the analysis. 
The project provided data on the 
number of quarters that students in 
the target group spent in each type 
of treatment. Number of quarters in 
each treatment was coded from 0 to 4 
in increments of .25 quarters. With 
this coding the unstandardized 
regression coefficient (slope) in 
the regression equation can be 
interpreted as the value on the out- 
come variable associated (i.e. 
added or subtracted) with one school 
quarter in treatment. 

Measures from School Records 

Results for a variety of measures 
obtained from school records are 
shown in Tables 9 through 12. 
Because students participating in 
various project components differ 
from non-participants on a variety 
of pre-intervention characterittics , 
it may be misleading to interpret , 
simple correlational results as 
effects of participation. The 
results shown in Tables 9 through 12 
are re sul t s of simpl e corr el at ional 
analyses of this kind. 

The association between number of 
quarters in cross-age tutoring and 
GPA and the as sociation between 
self-contained class and GPA are 
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both statistically significant. 
These results should be interpreted 
with caution because pre-tr eatment 
GPA data were not available in the 
schools where these interventions 
were implemented. Were dependable 
pre-tr eatment data available , they 
would be used as statistical con- 
trol s to probe the plausible 
hypothesis that this correlation is 
due to pre-existing differences bet- 
ween participants and non-partici- 
pants. 

Measures from the Spring 1982 Admin - 
istration of the SAES Student Survey 

The Student Questionnaire was 
administered anonymously to 7th and 
8th graders in two of the target 
schools (Pleasantville and Buena), 
7th graders in Vineland, and a ran- 
dom sample of 7 th, 8th, and 9 th 
graders in Pennsauken. Response 
rates and school level results are 
reported in" the overview^ of ""the "SAES ^ 
second interim report. 

Questionnaire booklets for target 
students were identified with a con- 
fidential survey number necessary 
for merging these survey data with 
information about participation in 
project components so that regres- 
sion analyses could be performed. 
Unfortunately, students who were no 
longer in treatment at the time of 
survey administration were not given 
the as signed booklets ; there were 
many missing data. Fifty (31%) of 
the target students were not admin- 
istered their assigned booklets. In 
addition, item non-response (a fai- 
lure of students to answer many of 
the questions in the booklets) made 
it possible to score only about half 
the scales, for each target student 
(sample mean=.49) . Biases of an 
unknown na t ur e may have resulted 
from this attrition and non-re- 
sponse, so the survey data are of 
much di minished value for 1982. 
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Because thesre were few students 
participating in either special 
clubs or cross-age tutoring and 
because of the previously mentioned 
missing data problem, student ques- 
tionnaire outcomes were not analyzed 
separately for these groups. Tlie 
same goes for the Pennsauken Self- 
Contained Cl^ss. 

The same regression approach that 
was described for the data from 
records outcomes v;as appl ied here. 
In cases where the association bet- 
ween a type of treatment and an out- 
come was significant, a multiple 
regression approach, utilizing step- 
wise regression v;as appl ied. ^ This 
procedure yiel ded an estimated 
regression coefficient (labeled "b" 
in the tables) that is adjusted for 
the contribution of other variables. 
Of the adjusted associations between 
treatment and outcome, only two 
apparently negative effects (the 
association between number of quar- 
ters in counseling and the score on 



2. The SPSS (Nie, Hull, Jenkins, 
Steinbrenner , & Bent, 1975) stepwise 
regression procedure with listwise 
deletion of cases with missing data 
was used. Stepwise regression 
essentially allows the computer to 
build a parsimonious statistical 
model of the criterion variable. In 
all cases, the computer run was set 
up so that it was possible for the 
following variables to enter into 
the model : race , grade level , age , 
school , number of quarters in coun- 
sel ing, community process, tutoring, 
special club and self-contained 
class. In the current situation, 
the purpose for performing the step- 
wise regression was to determine the 
value of an estimated "adjusted" 
regression coefficient and to test 
its significance. Parameters in the 
regression procedure were chosen to 
set the tolerance at .5 and the p to 
enter at .05 (see Nie et al . , 1975). 



the Rewards Index, and the associa- 
tion between participation in the 
School-Community Relations Class and 
Involvement ) remained. See Table 
13. 

An obvious interpretation of the 
analysis of Student Questionnaire 
outcomes would be that, the program 
did not have negative effects but 
neither did it help. Hie problem 
with such a conclusion is that the 
students who had left treatment at 
the time of survey administration 
were not administered an identifia- 
ble survey. This could mean that we 
essentially had no measurement of 
the proj ect's successes. (Alterna- 
tively It could mean we have no mea- 
sure of the project's failures!) We 
will have to v;ait until next year to 
be able to say anything definitive 
about program effects on outcomes 
measured by the Student' Question- 
naire. 

The Evaluation Design for 1982-83 

Finally, a non-equivalent control 
group design with appropriate num- 
bers of treated and untreated stu- 
dents has been agreed upon and 
implemented. The following number 
of ti'eatment and control students 
were chosen at the beginning of the 
1982-83 school year: 

Treatment Control 
Vineland 35 
Pennsauken 30 



Pleasantville 38 
Buena 34 



46 
36 
39 
35 



A Student Questionnaire pre-test was 
administered to all in October, 1982 
and the response rate was virtually 
100%. A variety of pre- treatment 
data from records have been provided 
for two pre-treatment years (i.e. 
1980-81; 1981-82). These data 
include standardized achievement 
test scores, the Tennessee Self-Con- 
cept Seal e scores, number of times 
suspended , GPA , at tendance , pr omo- 
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tion record, age, race, grade level, 
and sex. The data appear on initial 
inspection to be of high quality and 
contain few missing values. 

The on-site Evaluation Coordina- 
tor, now a veteran of the difficult 
position she took on, has made plans 
to administer the Spring liS3 Stu- 
dent Sui-vey in a more careful man- 



ner. 

The marginal results of the eval- 
uation of 1981-82 outcomes drove 
home the point about the need for a 
more intense level of intervention 
and a much stronger evaluation 
design. Students in the 1982~83 
cohort are staying in treatment much 
longer (most for the entire year)* 
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Table 1 





Tlic Original PDE Objectives of the 
and Actual Measure 


ElC-South Alternative Education Project 
5 of Achievement Employed 




Objective 


Measures Employed 




Goal 1 : Reduce 


(Prevent) Delinquency 


1. 


Raised self-esteem of students in Program classes 


SAES Student Questionnaire Postive Self Concept scale 


2. 


Decreased negative peer influence 


SAKS Student Questionnaire i^egative Peer Influence scale 


3. 


More mature attitude towaid drugs and persons 




4. 


More respect for public property (schools) and persons 




5. 


Awareness and "buying into" school rules by students and 
teachers 


Awareness possibly measured by Clarity of Rules scale 




Goal 2: 


Academic Success 


1. 


Improved teacher attitudes with regard to student's aca- 
demic success 




2. 


Improved physical environment 


1 ' ■ ' 



I 



I 
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I 
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Table 1 (Cont.) 



\% School Oi»lrict oponuiMiH to rr-rv.ilo»it ion and clwuijiinj' 



Co»\l ^ : KrJuce (Proven I ) SuH^-'VORi onH , ExpuU ioHH , Dropout a 

Jiuir.ntoiit bc)»(«(i on oxnininulion of policies 



?♦ School rrovido crontivo nnd dlvernifiiJ additional 
r«»HourcoH Tor dcnlinj; villi bohnviorn wcMihl normally 

r«H\ilt in MOKponHionn (oiiiHido of l\\v ucltool rosourcoR). 
•>op»cono in llio ucliool will coord inalv iht* reqnosiB for 
i»iul delivery of Wwhv Rprvices* 



3* Ch.mju' in ti»aihi«r nnd adninistralor altiludos iow«rd 
ii^inp, KUHpcniiion an a mrtlu>d of rpducin^; disruption 



Goal 5: With Howard to Dropout Specifically 



A« ZncriMfto acadomic buccc88 



- School gradoH 

- Siniuiard acliievcmcnt lost scores 

- S(»lf-rrp(irt j',rados from SAES Student Questionnaire 

- Self-report reading ability from SAES Student Qo<'S 
nnire 



5» tncrcaar self*catcon 



SAES Student Questionnaire Positive Self-Concept scale 



Goal 6: Effect Institutional Change 



1. Intfragrnvy cooporalion 

2. Coftrunity input into planning 



Documentation that ropresentatives from various age 
attend community process meetings and with what froque: 



Documentation that representatives from various age 
attend community process meetings and with what freque 



)• Yotith input into planning* and implementation 

4« Standardi lAt ion across a^encif*« nf KrnnHards of conduct 



Ri>ducc youth alienation and apathy 



SAES Student Questionnaire Alienation scale 



6% Positive community attitudes towards the Alternative 
Education Project 
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Table 2 

Revised Goals and Objectives of the ElC-South Alternative 
Education Project (Summer, 1982) 



Goal 1: To reduce/prevent delinquent acts by 10% for treatment group. 

!• To limit disciplinary referrals for classroom disruption by teachers, 

2, Improve/maintain attendance, 

3, Decrease out-of-school suspensions in Pleasantville by 10%; maintain 
levels in the other three districts, 

4, Increase positive peer influence, 

5, Establish an on-going process/structure to develop community stan- 
dards, resource use, problem solving, involving youth and other seg- 
ments of the community, 

6, Establish/increase opportunities for summer youth employment. 

Goal 2: To contribute to incr. eased academic success, - ^ ~ — „ _ _ 

1, Improve/maintain attendance, 

2, Improve the study skills of project students, 

3, Maintain high level of promoteion to next grade level. 

Goal 3: To bring about the adoption of proven alternative education programs 
and policies, 

1, To simplify school discipline policies to ensure comprehension by 
youth and parents, 

2, To expand and install concept of peer tutoring, 

3, To achieve adoption of one or more units of the Alternative Education 
Curriculum Guide by the participant districts, 

4, To ensure effective involvement of youth, 

5, To increase visibility of the New jersey Alternative Education Proj- 
ect, 

6, To increase the TA capacity for the State Board of Education in the 
area of alternative education and school climate. 
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Tnble 3 

Number of Students kcccivini^ hK\\ Type of Trfainicnt by the lac-South 
AUcrnativu liJuciitioii Project -- Broken Down by School and ijuartur 



School 









Vinclnnd 






I'ennsnuken 






Pkasantville 








iiuvna 




Treatment 


Qtr 
1 


Qtr 
2 


Qtr 
3 


Qtr 


Entire 
year 


Qtr 
1 


Qtr 


Qtr 
3 


Qtr 
4 


Entire 
year 


Qtr 
1 


2 


Qtr 
3 


Qtr 
4 


Entire 
year 


Qtr 
1 


Qtr 
2 


Qtr 
3 


Qtr 
4 


Entire 
year 


Alternative cd class 


8 


11 


22 


15 


23 


20 


19 


30 


31 


37 


15 


15 


33 


26 


39 


7 


8 


12 


11 


14 


Connunlty process 


0 


7 


0 


0 


7 


5 


0 


34 


30 


36 


0 


0 


0 


0 


0 


0 


0 


0 


0 


0 


Individual counseling 


5 


9 


18 


12 


25 


14 


13 


34 


31 


37 


0 


0 


18 


12 


23 


8 


9 


24 


22 


26 


Special club 


0 


0 


0 


0 


0 


4 


5 


2 


5 


8 


0 


0 




1 


1 


1 


0 


2 


3 


3 


Cross-age tutoring 


3 




2 


4 


6 


0 


0 


5 


6 


6 


0 


0 


0 


1 


1 


0 


0 


0 


0 


0 


Si'lf-contained class 












16 


16 


14 


14 


le 






















Tutoring 


0 


0 


0 


7 


7 


0 


0 


0 


1 


1 


0 


0 


0 


7 


7 


0 


0 


0 


0 


0 


Any treatment 










34 










42 










51 










;!() 


No treatment 










0 










1 










2 










3 
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Tabic A 

Avorar.o Number of Quarters tliat i:iC-Soutli Altcrnatlva 
Kducation Project Students Spent in ICacIi Type of Treatment 
1981-82 Sclwol Year — Broken Down by School 



Sch- 



Entire Project 



Vine land 



Pleasantville 



Treatment 


M 


SD 


N 


M 


SD 


N 


M 


SD 


:4 


M 


SD 


N 




\' i"» 


Alternative education 
class 


1.68 


1.40 


159 


1.43 


1.46 


34 


2 


.31 


1.37 


^t3 




1.16 


53 




1.56 


Community process 




.82 


159 


.20 


.41 


34 


1 


.58 


.83 


4? 


0 w 


0 


53 


0 


0 


Individual counseling 


1.39 


1.28 


159 


1.18 


1.14 


34 


2 


.10 


1.28 


43 


.57 


.72 


53 


2.07 


1.32 


Special club 


.U 


.50 


159 


0 


0 


34 




37 


.82 


43 


.03 


.'21 


.53 


.14 


.42 


Cross-age tutoring 


.16 


.58 


159 


.38 


.95 


34 




26 


' .66 


43 


.02 


.14 


53 


0 


0 


Self-contained class 


.36 


1.08 


159 


0 


0 


*34 


1, 


34 


|l.75 


43 


0 


0 


53 


0 


0 


Tutoring 


.09 


.29 


159 


.20 


.41 


34 




02 


1 .15 


43 


.13 


.34 


53 


0 


0 
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Table 5 



Representation of Non-Equivalent Control 
Groups Design Agreed to In .ugust, 1981 



Treatu.ent November April 

Population Starts Pretest Posttest 



4 target schools' September Yes Yes 

7th graders 

4 target schools' No Yes Yes 

8th graders 

4 Control Schools' No Yes Yes 

7th graders 

4 Control Schools' No Yes Yes 

8th graders 
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Tdble 6 



Numbers of Treatment and Untreated Comparison 
Students at Two Points in the 1981-82 School Year 



Total Number I^o 
Had Received Any 
Treatment 



Number of 
Comparison Students 



School 



^Through 



Jan '82 Mar '82 



Through 



Jan '82 Mar '82 



Vineland 
Pennsauken 
Pleas antville 
Buena 



18 
38 
19 
16 



33 
38 
50 
26 



16 

4 
34 
10 



1 

5 
3 



I 

I 

I 

I 
I 

I 
I 
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Table 7 



Zero Order Correlations Between 
Grades in Individual Courses for th 1980 - 81 
School Year - Vineland 



English 



Math 



Reading 



Sci ..uce 



Social Studie; 



English 
Math 
Reading 
Science 

Social Studies 



1.00 



.53 
1.00 



.76 
.06 
1.00 



.12 
.09 

-.26 
1.00 



.01 
.10 

-.22 
.49 

1.00 



•/ 
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Table 8 ' 

Zero Order Corr'ilations Between Background 
Characteristics and Amount of Various Treatments 
Received -' EIC - South Alternative Education Projects 



Treatment 



Days Absent Retentions In Age 
1980 - 81 Students History 



Race 



Grade 
Level 



Parental 
Education 



Tutoring 


1 A 

-.10 

(131) 


.04 

■ (151) 


■ (109) 


- 99 

(90) 


(95) 


-.21 

(40) 


School/Coniin 
Relations 


.12 

(131) 


-.03 

(151) 


.09 

(109) 


(90) 


• UU 

(45) 


- in 

(40) 


Community 
Process 


.18 

(131) 


.24 

(151) 


.23 

(109), 


.47 

(90) 


.13 

(95) 


-.22 

(40) 


Counseling 


.16 

(131) 


.00 

(151) 


.18 

(109)^ 


.28^ 
(90) 


-.10 

(95) 


-.04 

• (40) 


Special 
Club 


'.23 

(131) 


.18 

(151) 


.14, 

(109) 


-.02 
(90) 


• .05 

(95) 


r.26 

(40) 


X-age tutoring 


-.02 

(131) 


.05 

(151) 


.07' 

(109)1 


.13 

(90) 


.00 

(95) 


-.05 

j(40) 


Self-contained 
Class 


.28 

(131) 


.34 

(151) 


.48' 

(109)| 


.35 

(90) 


.40 
(95) 


-.33 

1(40) 

i 



So 



te: tabers in patenttases are n's. The n's vary duej to differential degrees of .Issinj data. 



Table y 



Regression of Outcome Measures on Degree of 

Participation in Treatment 
Components Operating in All Four ElC-Schools 





Days of .non- 
attendance 




GPA on 
1-5 sea 


a 

le 


Number of times 
suspended 


Quarters in 


b 


N 


b 


2 

r 


N 


b 




School-community 
relations class 


-1.36 .01 


143 


.07 


.02 


154 


-.32 


.02 142 


Counseling 


-1.27 .01 


143 


.04 


-.00 - 


154 - 


--.02 


.00 - 142 - 



Note.. Each b is the unstandardized regression coefficient in a separate equation 
regressing the outcome variable on the treatment variable. None of these regres- 
sions is significant. 
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Table 10 



Regression of Outcome Measures on Degree of 
Participation' in Treatment Components 
Operating Only in Vineland and Pennsauken 





Days of non- 
attendance 




GPA on a 
1-5 scale 


Number of times 
suspended 


Quarters in 


b 


2 

r 


N 


b 


2 

r 


N 


b 


N 


Conimunity process 


-1.49 


.01 


71 


.08 


.01 


75 


-.04 


.01 73 


Cross-age tutoring 


-4.20 


.04 


71 


.23* .06 


75 


.00 


.00 73 



Note . Each b is the unstandardized regression coefficient in a separate equation 
regressing the outcome variable on the treatment variable. 



*p<.05. 
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Table 11 



Regression of Outcome Measures on Degree of 
Participation in Self-Contained 
Class Operating Only in Pennsauken 





Days of .non- 
attendance 


GPA on a 
1-5 scale 


Number of times 
suspended 


Quarters in 


b r^ N 


b N 


b 


r2 N 


Self-contained 
class 


-1.68 .02 39 


.20** .19 41 


-.04 


.03 39 



Note . Each b is the unstandardized regression coefficient in a separate equation 
regressing the outcome variable on the treatment variable. 

**p<.01. 
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Table 12 



Regression 


of Outcome Measures on Deg^c 
Operating Only in Pennsauken, 


e of Participation 
Pleasantville and 


in Special Clubs 
Buena 




Days of non- 
attendance 


GPA on a 
1-5 scale 


Number of times 
suspended 


Quarters in 


b r^ N 


b r^ N 


b r^ N 


Clubs 


.91 .00 111 


.04 .00 120 


-.48 .01 108 



Note . Each b is the unstandardized regression coefficient in a separate equation 
regressing the outcome variable on the treatment variable. 
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Table 13 



Rt'^rfsslon of 1982 Studt-nt ljuL*st ionna i rc Scale 
find Seluctoil Itam Scores on l)ei',rce of I'art- 
icip;ilion in Variouti Treatment Activities— 
Ney Jersey 



Qnnrtcrs in Alt. Quorters In Con- Qunrtcrt. In 

Kducntlon Closs munity Procoss Ctumsf 1 Iny; 





raw 




adjusted 


raw 




adjusted 


raw 




^djus led 




b 


N 


b N 


b 


N- 


b 


K 


b 


N 


b r- 


Parontnl cnphasis on 
cducat ion 


-.08 


AO 




-.10 


11 


— 




-.01 


AO 




Attachment to parents 


-.06* 


lOA 


-.05 86 


-.22** 


A7 


-.05 


86 


-.lO't* 


lOA 


-.30 20 


Kc{^ativc peer influence 


.00 


97 




.18** 


A6 


.OA 


53 


.03 


97 




Attachment to school 


.00 


65 




-.09 


21 


— 




-,0A 


63 




Belief scale scored 
positively 


-.03 


3A 


— 


-.16 


21 






-.OA 


3A 


— 


Interpersonal conpe- 
tency 


.01 


3A 




-.33** 


9 


-.16 


21 


-.01 


3A 




Positive self-cbnccpt 


.03 


28 


__ 


.02 


6 


— 




-.00 


28 


— 


Self-reported delin- 
quency 


.01 


88 




•.>20** A2 


-.03 


71 


-.03 


88 




Solf-repcrtod drug 
use 


.Oi 


88 


— 


.3A** 


A5 


-.02 


62 


.06 


88 


— 


Serious delinquency 


-.01 


91 




.13* 


A5 


.03 


7A 


. 01 


91 




Punishment index 


-.03 


75 


— 


-.03 


27 






-.03 


75 


— 


r>evards index 


-.07* 


75 


-.01 5A 


-.OA 


27 







-.10*'' 


75 


-.10'"* 5A 


Victirti ration 


-.Oi 


69 




-.09 


2A 


— 




-.06* 


69 


-.03 5A 


Invalidity scale 


-.OA 


3A 




-.05 


a 


— 


- - 


-.06 


33 





School effort 


-.02 


102 


— 


-,01 


A8 






-.01 


lor 


-- 


Practical knowledge 


.11 


31 




-.25 


8 


— 




-.05 


31 




Internal-excernal 
control 


.06 


3A 




-:o8 


8 


— 




. 02 


3A 




Alienation 


-.01 


3A 


— 


-.07 


8 






.00 


3A 


— 


Self— reported grades 


— . U J 






-.02 


A8 


— 




-.01 


107 




Reading ability self- 
rating 


.11 


99 




.09 


A5 


— 




.OA 


99 





Days of .school cut last 
four weeks 


-.06 


102 


— 


-.01 


A7 






.05 


102 




How often cut one or 
core classes 


-.06 


108 




.A7 


A8 






.18 


108 




Educational expectations 


.lA 


105 




,A3 


A6 






.06 


105 




Did student v.*ork for 
pay last week? 


-.01 


106 




.02 


A8 






-.03 


106 




Regular part-tine or 
full-tine job 


-.06 


lOA 




.07 


A6 






-.03 


lOA 




Suspended from school 
this term 


-.02 


56 




.10 


19 






-.03 


56 




School ncnattendance 
index 


-.05 


102 




.28 


A7 






.08 


102 




Rebellious autonony 


.02 


32 




.3A* 


8 


.18 


31 


.09 


32 




Involvenent 


-.07** 98 


-.03** 78 


-.15** 


A7 


-.05 


78 


-.06** 


98 


-.01 78 



Note . Fir.ures In the colur^^s entitled "raw" nre unst.mdardi zed roRrossIon coef f IcIcnt^^ f(^T 
rtir.plo rL•^,^L•f^sIon^; of each oittrome varlahli- on t.ach of the i ntW|W«fnk'nL varlahlos (I.e. Quar- 
ters in All. Ed. class. Quarters In Community Proeess» and (^inrters in Cnunrtel In^) . 
!CaTbers in "adjusted" coluns^fi are unstandardir.cd roRressIon coefficients from nultlple 
regressions. wiiore» in addition to the independent variablt' of interest (i.e. the trentneiit) 
control varlablifi were Introihicecl into the eqiiation. The purpose of such a procccl 
Is to attempt to ctintrol for the effects of pn-exIstlnK cliaracter I sti c?< nf tlu' stiulents. 

The control variables utilized were the following; grade level, race, ape. school of 
attendance, participation In other treatments (I.e. counseling, clarts, tutoring, clubs, 
cormunlty process). Of these, only those whose contribution to a reRresrilon eqiiatlon 
was statistically significant (i.e. p .05) with a tolerence of .5 were utilized. 



*p <.05 
**p <.01 



Appendix A: Guiding Principles of the ElC-South Alternative Education 

Project 



Working Theory of Delinquen cy 

Delinquency is a means of expression both in a protesting sense and 
as a means of getting attention (youths don't know how to get around in 
the system in a conventional way). Many things contribute to it. 

1. Self-esteem — "Delinquent acts are the only thing I can successfully 
accomplish. This is the only thing I can be proud of." 

2. Family relations — no communication, guidance, supervision, and youths 
raise themselves. Family does not view itself as a means for provid- 
ing .upnort for the total growth of the youths (provide food, shel- 
ter, and clothes, that's all) and in some cases is not a cohesive 

unit. Negative contacts with community institutions > powerless- 

ness to do anything about things other than food, shelter, and cloth- 
ing. 

3. Boredom — Youths aren't challenged in school. Curricula is irrele- 
vant. Youth activities provided by the community are irrelevant and 
they are "provided" instead of decided and managed by the youths. 
This is a great and easy cop-out for the youths. They don't know how 
to get around in the conventional system and don't find out how to; 
thus they cop-out. Lack of programs to participate in. Transporta- 
tion is an obstacle. Parents are usually uncreative in thinking up 
activities to sustain the family unit. 

4. Negative peer and adult contacts — peers and some adults positively 
responding to delinquency which creates more delinquency. 

5. Poverty — no money which leads to delinquent acts which are gratify- 
ing. Youths don't know other ways of getting money. 

6. Drugs and alcohol — to get these youths need money which means commit- 
ing delinquent acts to get it. Once they have the drugs and alcohol, 
youths do crazy (delinquent) things. Drugs and alcohol are delin- 
quent, and the youths have immature attitudes toward their use. 
These contribute to academic failure because youths are spaced out in 
school and not responsive to any learning which may take place. 

7. Depersonalized victims 



This appendix is drawn from material prepared by project personnel in 
completing the PDE plan. It is reproduced here with only very minor 
editing. 
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- Youths don't have a sense of ownership of public property and do 
not feel a need for preserving the property 

- don't know their "neighbors" so hurting then or their property is 
not personalized 

8. Lack of jobs leads to no money which youths want (they don't know 
how to participate in a conventional sense). Transportation is 
sometimes an obstacle to employment. 

9. Lack of uniform standards of conduct and the application of those 
standards (vague school rules , without community input ) . 

-inconsistency in rules and the consequences for breaking a rule 
between institutions. 

-d.ouble standards (youths can't smoke but adults can) 

-youths as consumers of these institutions, do not have input in 
the decision-making which governs them 



Youths experience academic failure - because : — . _ _ 

Youths enter school with low self-esteem 
Schools are not responsive to individual needs: 

a. Youths are unable to fit into the system due to measurement (used 
as a means of tracking, labeling, not as diagnostic tools) and 
cultural differences which lead to dissaffected youth not "buying 
into" the system. 

b. Youths don't feel cared for because of teachers, principals and 
school policies. 

Students have no input, particularly in curriculum, (thus it is bor- 
ing), discipline, and policy making. Youths end up not caring 

Teacher attitudes: (hopelessness, cynicism, etc.) 

a. Teachers don't expect youths to achieve which leads to youths not 
being challenged. In many cases, the potential is there, but no 
motivation to achieve. 

b. Lack of communication between teachers and adminstration directly 
affects teacher ownership (they do not feel they have ownership). 

Parental contacts with school are for negative reasons and "they 
don't want to hear it^'* Parents aren't supported by the school or 
encouraged to support their children. 
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Youths not feeling su^ by teachers and parents leads to teach- 
ers not feeling suppoxic principals, superintendents, board, com- 
munity, and student'iS pa.. leads to principals not feeling sup- 
ported by superiors, boaro munity, teachers (depending on the 
school), and parents (deper. on the prinicpal) leads to superin- 
tendents not feeling suppoi: by: board, community leads to boards 
not feeling supported by comm.. ity (superiors) leads to community not 
ge 1 1 ing invo Ived . 

The above problems die a slow and painful death and there is no 
resolution. 

Principals 

a. Are responsible for insuring that youths come to school. 

b. Carry out the tracking, and labeling procedures because they 
impact scheduling . 

c. Approve the curriculum/development implementation and can 
change the whole curriculum- 

d. Strongly influence budgets which don't end up buying needed 
books, sponsoring field trips, and providing custodial care 
for after-school activities. 

e. Are responsible for teacher placement. ("Good ones" end up in 
"bad" classes); consequently, teachers don't want to perform 
well because there are limited rewards, if any. 

f. Carry out disciplinary responses sometimes inconsistently due 
to others ' influences (board, superintendent, politicians ) . 

g. Are responsible for insuring that youths come to school, but 
the various responsibilities deem that they leave coming to 
school up to youths and, parents. 

h. Designate responsibility to the guidance departments and the 
local court (15 dollar fines for not sending youths to 
school). Guidance department are busy with "successful" 
youths and pass the problems on to truant officers. 

Boards of Education superintendents (and community) are not planning 
educational goals and plans, budgeting for the whole district, and 
the small minority of "bad youths" get lost in the plans. 



Classified students 



-Youths who are legally classified present another significant prob- 
lem in the theory of student academic failures. These youths are 
very often scapegoats for an education system which uses labeling 
and tracking techniques to rid itself of these students who have 
not conformed to the schools social and educational "norms." 
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-Teachers are not trained to adequately deal with the multi-faceted 
handicaps of the classified student. Insufficient teacher train- 
ing and limited experiences with this type of student lends 
itself to low eacpectations from teachers and no motivation on the 
students^ part. 

Most frequently used levels of classification for the special 
education student- 

-neurologically impaired 
-perceptually impaired 
-emotionally disturbed 
-socially maladjusted 

Theory; Violence and Vandalism 

1. Self-esteem — "Delinquent acts are the only thing I can successfully 
accomplish. This is the only thing I can be proud of." 

2. Boredom — Youths aren't challenged in school . 
Curricula is irrelevant. 

Youth activities provided by the community are irrelevant and they 

are "provided" insteac of decided and managed by the youths. 

This is a great and easy cop-out for the youths. 

They don't know how to get around in the conventional system and 

don't find out how to; thus' they cop-out. 

Lack of programs to participate in. 

Transportation obstacles. 

Parents not being creative in thinking up things to do or making 
youths responsible in the home. 

3. Drugs and alcohol — to get these' youths need money which leads to del- 
inquent acts to get it* Once they have the drugs and alcohol, they 
do crazy (;jielinquent) things. Drugs and alcohol are delinquent, and 
the youths\have immature attitudes toward their use. These contri- 
bute to academic failure because youths are spaced out in school. 

4. Depersonalized victims 

- Youths do not have a sense of ownership of public property and no 
stake in preserving the property 

5. Negative peer and adult contacts — peers and some adults positively 
respond to delinquency which leads to more delinquenct acts. 

6. Lack of uniform standards of conduct and the application of those 
standards (vague school rules, without community input). Inconsis- 
tency in rules and the response to breaking a rule between institu- 
tions and double standards (youths can't smoke but adults can). 
Youths , as consumers of these institutions , do not have input into 
the rules which govern them. 

7. Gap not bridged: transferring the idea of private ownership to public 
property so that youths feel as if they own it too. This leads to a 
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need for 3'ouths to become involved in and responsible for ownership 
as a constituency' of the community in partnership with others, spe- 
cifically peer and adults. 

8. Rage — "I'm not v;orth anything!" These are often expressions of anger 
at not being recognized. These expressions may lead to attacks on 
teachers (and others) who become personalized victims of the rage. 
Many demeaning things have accumulated over time and culminate with 
the expression of rage, hostility. 

Theory: Absenteeism 

1. School becomes a negative place to be. 

a. The differences between youths (skin color, dress & other) are 
apparent to youths/teachers- and some youths get a lot of negative 
responses . 

b. Language barriers between: Hispanics & English, Black dialect & 
English. 

c. Problem with curriculinn and achievement leads to poor testing 
skills which leads to being "labeled." 

d. Parents transfer their negative feelings about-the school (see- 
academic failure. No. 5) to their youths. 

e. Boredom — Youths are not challenged in school. Curricula is 
irrelevant. Youth activities provided by the community are 
irrelevant and they are "provided" instead of decided and managed 
by the youths. This is a great and easy cop-out for the youths. 
They don't know how to get around in the conventional system and 
don't find out how to; thus they cop-out. Lack of programs to 
participate in. Transportation obstacles. Parents not being 
creative in thinking up things to do. 

f. Students have no input, particularly in curriculum (thus it is 
boring), discipline, and policy making which leads to youths not 
caring. 

g. Teacher attitudes : (hopelessness, cynicism, etc. ) 

1) Teachers do not expect youths to achieve which leads to youths 
not being challenged and boredom becomes commonplace! 

2) Lack of communication between teachers and administration 
directly affects teacher ownership (they don't feel they have 
ownership too). 

2. Due to family structure (responsibility for babysitting while parents 
work), youths have to work (migrant workers). Migration of the 
family interferes with going to school. Education is not valued. 

3. Social Success 
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a. Youths who go to school and are not able to communicate with stu- 
dents, teachers, parents (no interaction between them) but v;ho 
want social success, often find it with others who aren't in 
school • 

b. Can't handle all that's thrown on them (not adults yet). Moon- 
lighting in jobs and can't be succesful in both school and work. 
The job wins out (its more rewarding). They want immediate grati- 
fication so they get jobs and school is not an immediate gratifi- 
cation. 

Theory : Suspensions , Expulsions , Dropouts ; 

Expulsions don't happen very often because it is a difficult process 
(due process requirements). Expulsions only happen in very clearcut and 
serious circumstances. Intervening at an earlier stage prevents expul- 
sions . 

Suspensions occur because they are a very easy way of coping (getting 
rid of) with disruptive students. Teachers pressure administrators to 
suspend disruptive students. They don't know what to do with youths. 
Suspensions, as a consequence (punishment), are based on the assumption 
that the youths want to be in school in the first place. In reality, 
being suspended can be a reward to some youths. 

Guidance counselors work loads don't allow the time to deliver counsel- 
ing services that disruptive youths need. 

Drop-outs (push-outs? ) 

Youths drop out because: 

1. They are not academically successful 

2. They have low self-esteem and no sense of worthiness 

3. They want to earn money 

4. Of boredom 

5. Of family relatives, who appear to be "making it." 

6. Of drugs and alcohol/problems. 

OR disruptive youths are counseled out the door. Which means that 
some students are not encouraged to perform at any level. 

They dont' fit in any program and therefore, in subtle waysj are 
pushed out the door. 
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Theory; Effect Institutional Change 
In view of perceptions of fact and the stated problem definition, the 
following hypotheses have been drawn: 

1. Academic success (positive experiences at various levels of abil- 
ity), attendance rate , suspension, expulsion, dropouts , truancy 
are not subject to impact by schools alone. 

2. The school system in any given community, is so reflective of the 
attitudinal make-up of that community that the schools' effective- 
ness in implementing change in areas such as violence, vandalism, 
suspensions and expulsions is limited to remediation at best, 
without a change in the ct^mmunity attitudes that the school system 
reflects . 

3 . Schools , municipal governments , juvenile service providers , and 
independent service groups are generally operating within their 
own spheres to alleviate causal factors such as permissiveness in 
society, lack of respect for authority and property, boredom, and 
drug and alcohol abuse. This is primarily dealt with on a case by 
case basis; this approach results in a haphazard individual remed- 
iation program, with little or no long range preventative effect. 

4. Youth alienation and sense of powerlessness on the part of the 
community; what we commonly refer to as "apathy" -in terms of vot-- 
ers, community groups, PTA and school councils, etc., is more 
likely a result of the sense of powerlesness than a demonstration 
of the attitude of the community in general. 

5. This "apathy" is a direct result of a feeling of alienation from 
the very institutions that have most direct impact on the commu- 
nity: schools , local government , social service providers . The 
implementation of a variety of programmatic solutions has created 
a confusing set of standards of behavior for children, parents, 
and the community. These standards are not reflective of what the 
community desires, although the community feels powerless to 
change them. 

6. A problem-solving process must be designed and implemented which 
will allow the community direct input into analyzing the causes of 
juvenile problems in schools, home and community. These causes 
must be sufficiently defined to allow for programmatic application 
and solution; the process must involve all agencies charged with 
effecting the solutions along with the community; the solutions 
must be a cooperative effort so that schools, government, service 
providers, and the community are not working at cross-purposes, as 
antagonists, or in competition with each other. 

a) The process in each community must be credible, and subject to 
an intensive public relations campaign. 

b) All resources, public and private, should be brought to bear 
upon the causes through systematic youth, parent, community and 
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educational programs, vocational training and placement, 
counseling, and individual service programs. 



7. At a minimum, the process should result in: 



a) a standard of conduct that the community deems appropriate in 
its schools and throughout the community; 

b) widespread dissemination of that standard and its community 
basis, together with a rewards/punishment system; 

c) fair and uniform application by schools, police courts, and 
other agencies charf^ed with these duties by the community; 

d) positive attitude development throughout the entire school cur- 
riculum; and 

e) on-going assessment by the community of its standards. 



Community Process 



A. Advisory councils are not necessarily what the project is hoping 
to formulate. Advisory councils are usually composed of: 

1) Most times the same people who always get involved. 

2) This means only 35-40% of any given community is directly 
involved. 



B. Expectations of community process 

-Create climate for change (working with formal /infor^nal leader- 
ship) 

-Don't ask for big committee or time 
-Ask for suggestions/recommendations 

-Hopefully they will start thinking process for what the community 
wants to happen 

-Inter-agency cooperation 

-Want change to occur from the inside 
without threatening anyone's territory 

-Communication extremely important 

-Agree with the "Rock-Throwers" 

-Ask them to do something about their complaints 
C. Kinds of people/organizations to invite to meetings 
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-Schools 

-Municipal Governments (Elected and appointed representatives) 
-Church 

-Youth ("good" and "bad") 
J -Business/Industry (Merchants Asso., Chamber of Commerce) 
-Kiwanis/Lions/Jay Gees 
-PTO/PTA's 

-Social service delivers - paid professionals delivering ser- 
vices for youth 
-Ordinary People 
-Police 

-Courts (municipal , courts ) 
-Senior Citizens 
-Families 

-Social /Recreational 

-Juvenile . Conference Committee (citizens, meetings, as needed, 
to handle 1st infraction for youths: punishment, etc., to be 
erased from youth's record) 

-Knights of Columbus 

-Firefighters (volunteer ) 
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Jewish Vocational Services Alternative Education Project: Evaluation Report 

M. S. Cook 



Abstract 

The Jewish Vocational Services Alternative Education Project 
developed and impl emented three interventions. The Milwaukee Youth 
Employment Center (HYEC) counseled dropout youth and attempted to 
place them in employment . Tlie Return Center , operated in coopera- 
tion with the Milwaukee Public schools, assessed and referred to 
alternative educational programming youth \jho were contempl ating 
dropping out, or who had already dropped out and wished to re-enroll 
in formal education. The Job Score class was a regular high school 
course developed by MYEC staff to teach employment skills to youth 
at risk for dropout. 

An experimental evaluation of the MYEC program indicates that it 
was not successful in increasing the employment opportunities of its 
clients. Subsidiary analyses suggest that in general , clients did 
not receive many services , although remedial education provided 
through the proj ect probably had positive effects. Evaluations of 
two proj ect components — the Return Center and Job Score cl as- 
ses — were not completed due to the early termination of the project. 

In contrast to some recent reports, regression analyses suggest 
that employment causes little, if any, pychological harm to youth 
who have already dropped out of school. Returning such youth to 
school may result in some negative outcomes. 

The Job Score curriculum may be appropriate for use and evalua- 
tion by others interested in providing employment counsel ing to 
youth . It is recommended that prevention proj ects be allowed to 
follov; a Program Development Evaluation cycle that provides more 
time for program refinement and testing than was available to this 
project. 



Overview 

Thft Jewish Vocational Services 
Alternative Educational Proj ect was 
an attempt to provide alternative 
services to students who had dropped 
out of the Milwaukee Public Schools, 
or were at risk of dropping out . 
Jewish Vocational Services (JVS) is 
a large agency with a 40-year his- 
tory of operation in Milwaukee, and 
it has particular expertise in 
employment problems faced by special 
populations such as immigrants , 
youth, and the handicapped. 



In the two year history of the 
alternative education project, three 
distinct interventions were devel- 
oped and implemented by JVS : 

(a) the Milwaukee Youth Employment 
Center, a job counseling and place- 
ment center for dropout youth; 

(b) the Job Score class, an employ- 
ment counseling curriculum for use 
in the regular public school curri- 
culum; (c) and the Return Center, an 
assessment and referral center for 
youth seriously at risk of dropout. 
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4. Increase feelings of personal 
power ( internal control ) . 

5. Provide youth with income 
through employr»ent . 

6 . Decrease youth ^s time spent 
nonproductively . 

7. Return dropout or potential 
dropout youth to more adequate 
alternative education programming. 

8. Develop and test a work-re- 
lated, competency-based high school 
curriculum. 

History and Organizational Context 
of the Project 

The JVS Alternative Education 
Project was plagued with implementa- 
tion problems from its very incep- 
tion. Many of these will be 
detailed below in conjunction with a 
description of the particular inter- 
ventions. Presented here are gen- 
eral problems that affected the pro- 
ject as a whole. 

Just as the project was funded in 
December of 1980, JVS underwent a 
severe organizational shake-up. The 
organization was undergoing federal, 
state, and local audits as a result 
of alleged embezzlement of funds. 
At the same time, federal cut-backs 
in social service funding had 
resulted in layoffs of JVS union 
staff. A new executive director was 
hired to oversee the stabilization 
of the organization during its fis- 
cal crisis* Thus the project was 
born in an organizational climate of 
crisis and distrust. 

The layoffs in staff continually 
subjected the project to union 
demands for pr ior ity in hiring , 
regardless of qualifications. Line 
staff still on the job did not trust 
the organization, and resorted to 
strikes or strike threats to press 
their demands. 



JVS had a history of poor rela- 
tions with local CBOs, social ser- 
vice agencies , and the Milwaukee 
Publ ic School s . All of these agen- 
cies were eventual ly required to 
serve as referral sources and inter- 
vention sites. Referrals therefore 
were slow to develop, and negotia- 
tions for cooperative work on parti- 
cular interventions were long and 
arduous. 

An early error in staffing the 
project occcurred when* JVS hired a 
non-jys person to direct the pro- 
j ect . This did not turn out well 
because the project director never 
felt himself to be part of the 
organization, or particularly 
accountable to it. The project 
director, being an outsider, was not 
invested in the project as a whole, 
but only in those parts that he felt 
he was hired specifically to attend 
to. Thus, for example, the school- 
related" par^ts^of ''the" project^ CtHat 
OJJDP was especially interested in) 
were given short-shrift because the 
project director felt that their 
management was beyond his purview. 

JVS proposal writers, too, may 
have promised more than could be 
expected of the project. For exam- 
ple, in the original grant proposal, 
JVS promised to serve 750-800 youth 
in the MYEC job counseling program 
during the first year — it actually 
served fewer than 350, and only 79 
of these received the entire treat- 
ment. Similarly , the proposal said 
it would serve 40-100 youth in the 
Return Center during the initial 
year. As it turned out, the Return 
Center did not begin until the sec- 
ond year of the project. 

JVS and the Sponsor 

An additional organizational con- 
flict lay in JVS^s relationship with 
its federal sponsors. JVS adminis- 
trators failed to quickly or seri- 
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ously deal with the concerns that 
OJJDP had on budget, management, and 
program implementation. Two events 
that ultimately may have contributed 
to a decision not to continue the 
project for the third yeuL will 
illustrate the point. First, in 
August, 1982, the project was still 
unaware of its continuation status 
for the third pr oj ect year . JVS 
management chose to address this 
ambiguity by not sending anyone to 
the national conference — an OJJDP 
requirement . Inexplicably nobody at 
OJJDP or at Johns Hopkins was even 
notified of the JVS decision not to 
appear at the conference. They sim- 
ply did not show up. Hurried phone 
calls, and the reminder that appear- 
ance at the conference was an 
already-incurred contractual obi iga- 
tion, produced a JVS del egation a 
day late. 

Second, after the JVS staff 
appeared, they were unable to suffi- 
ciently engage in program develop- 
ment efforts — the purpose of the 
conference — because they were still 
trying to prepare a budget for the 
third year proposal . Despite a 
clear requirement that all projects 
submit a third year budget that was 
30% smaller than the previous year, 
the JVS third-year proposed budget 
was 20% larger. Project staff spent 
a great deal of the time at the con- 
ference deciding whom they were 
going to lay off when they got back. 
Such thoughts are not conducive to 
program development efforts. 

These JVS organizational problems 
subj ected the proj ect to continuing 
upheaval over interpersonal rela- 
tions , interorganizational rela- 
tions, labor relations, and bugetary 
management. Little organizational 
effort remained to focus on the 
interventions themselves. 



The Interventions 

Milwaukee Youth Employment Center 
The Milwaukee Youth Employment Cen- 
ter (MYEC) was the core intervention 
of the JVS project. MYEC was a "job 
readiness" training program, model ed 
after the Jobs For Youth, Boston, 
program. Clients were trained in 
the skil 1 s necessary to f ind , 
secure, and keep employment , and 
were referred to jobs identified by 
the project. Youth .served were 16- 
and 17-year-old, unemployed, offi- 
cial high school dropouts from the 
greater Milwaukee area. 

The MYEC program consisted of 
three units: (a) a counseling unit 
that trained youth in the job readi- 
ness skills; (b) an Educational Ser- 
vices unit that provided competency- 
based individualized instruction to 
those clients needing or desiring 
it; and (c) an Employer Services 
unit that located jobs available, 
and acted as the employers' advocate 
within the program. 

The MYEC project went through two 
phases: March to December 1981, and 
January 1982 to the end of the pro- 
ject in November, 1982. In the 
first phase of the project, job 
readiness skills were identified 
(based largely on the Jobs For 
Youth, Boston, model), and youth 
were assigned to individual counse- 
lors for training in the skills. In 
the second phase of the the project, 
more detailed skills were identi- 
fied, criteria for successful mas- 
tery were outlined, and a structured 
series of workshops were begun which 
all clients had to attend. 



There were three "core" job read- 
iness skills that had to be mastered 
before one was referred to a job: 
job appl ication, job interviewing, 
and job retention. In addition to 
these core skills, clients were also 
judged for job readiness by their 
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counselor based on their appearance, 
punctuality, dependability, and 
attitude. 

In phase one of the project, 
clients were assigned to an indivi- 
dual counselor at intake, who 
trained them in the skills, and 
referred them to a job when the 
counsel or f el t that the cl ieat was 
job ready. In phase two, the skills 
were taught in a structured workshop 
format, and attendance at the work- 
shops v;as generally regarded as suf- 
ficient evidence of job readiness. 
Each client was also assigned a 
counselor. An individual coiild 
retake a workshop, or receive indi- 
vidual help from the counselor if it 
was apparent that he or she had not 
mastered a requisite skill. 

In both phases cl ients al so 
received general counseling "as 
needed." Also, those youth who 
decided to upgrade their academic 
skills (either by their own wish, or 
at the suggestion of the counselor) 
were referred to Educational Ser- 
vices. 

Educational Services provided 
competency-based individual ized 

instruction in basic skills (reading 
and math), advanced instruction in 
G. E.D. subj ects , and functional 
vocational training including typ- 
ing, cashiering, and driver's educa- 
tion. Educational Services was an 
optional part of the MYEC program. 
Youth who did not need further edu- 
cational assistance did not receive 
its services. 

The Employer Services unit was 
responsible for developing jobs 
(i.e., locating job openings and 
creating good will among employers) 
so that positions to which MYEC 
clients could be referred would be 
available. 

An important element of their 



responsibility was ensuring that 
employers were satisfied with the 
quality of ^^YEC referrals. A criti- 
cal part of the program (particu- 
larly during a recession) was the 
ability of Employer Services to have 
a* job bank available from which MYEC 
youth could choose appropriate 
employment. While the number of 
avail abl e positions fluctuated dur- 
ing the time the project operated, 
there were usually jobs available 
for the youth to be referred to . 
The mor e common pr obi em was the 
quality of the available jobs. Many 
were part time, and virtual ly al 1 
were very low level entry jobs such 
as counter clerks, dishwashers, res- 
taurant workers , and maintenance 
personnel . 

Youth who were referred to jobs, 
but did not land them, were continu- 
ally referred until they found 
employment or dropped out of the 
program". Youth ~who landed ^a"" job 
were followed up by their counselors 
at irregular intervals to see how 
they were doing (although this did 
not happen as often as project plans 
called for). Youth having diffi- 
cul ty on the job were encouraged to 
come back to MYEC for additional 
counseling or training. Youths who 
lost jobs they had been placed in, 
either because they quit or were 
laid of f , could be re-referred if 
the circumstances warranted it. 
Employer services representatives 
checked with the employers of youth 
who lost jobs in order to assess the 
difficulty the youth had with 
employment. This often led to addi- 
tional counseling for the youth. 
For exampl e , a youth laid of f for 
tardiness would not be referred to 
another job until some judgment had 
been made that tardiness would not 
again be a problem. 

As al luded to earl ier , the pro- 
gram began with an emphasis on one- 
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to-cne counseling provided to the 
clients by their assigned counse- 
lors. In the fall of 1981, the pro- 
ject director became dissatisfied 
with this arrangement, believing 
that the counseling provided to 
clients varied too greatly depending 
upon the counselor. In addition, 
the Employer Services division of 
the MYEC program compl ained about 
the "j ob readiness" of the gradu- 
ates. Employer Services was receiv- 
ing compl aints from employers about 
the qual ity of ref erral s from the 
program, and was frustrated by the 
number of cl ients who accepted jobs 
and then quickly quit them. 

In order to insure that all 
clients received training in the 
three identified job readiness 
skills, and to gain some quality 
control over program del ivery , a 
workshop format for training was 
developed and implemented in January 
of 1982. This change initiated 
phase two of the project. In the 
workshop format, MYEC clients were 
still assigned to a personal counse- 
lor, but that personal counselor was 
no longer solely responsible for 
providing the job readiness train- 
ing, and certifying the client as 
job ready. Instead, each client 
went through a structured series of 
workshops. Workshops were held at 
set times each week, and cl ients 
progressed through the workshops in 
serial fashion. MYEC clients first 
attended an orientation workshop, 
then one on completing job applica- 
tions, follov/ed by a workshop on 
interviewing, and one on job reten- 
tion. 

Satisfactory compl etion of the 
four workshops resul ted in certifi- 
cation as job ready. Although de- 
termination of "satisfactory perfor- 
mance" was still largely a judgment 
on the part of the counselors con- 
ducting the workshops (except for 
the job application skills workshop 



where a post-test job application 
had to be completed with 80% accu- 
racy in order to be certified) , the 
v;orkshop format was an improvement 
over the previous procedures because 
it insured that every cl ient was 
expo sed to th e fundamental s of ea ch 
skill believed to be necessary for 
empl oyment succes s . 

The switch to the vzorkshop format 
also improved the program for a sec- 
ond reason: it forced project per- 
sonnel to commit to paper exactly 
the skills and information necessary 
to meet the cr iterion of "j ob 
ready . " In order to write a work- 
shop that would be presented to all 
'clients, and could be facilitated by 
any of the counselors, project staff 
were forced to come up with a set of 
curriculum materials that specified 
the information to be taught, and 
the methods and techniques to be 
utilized. The technical assistance 
contractor (Polaris) provided train- 
ing in group facilitation skills to 
the staff specifically for the pur- 
pose of strengthening the workshops. 
Such training does not insure that 
the staff acquired the necessary 
skills, but it at least forced the 
proj ect to engineer into the work- 
shop specific activities aimed at 
the job readiness criterion, and 
gives us more confidence that all 
clients received instruction in the 
j ob readiness skil 1 s . 

At the same time that the work- 
shop format was developed and imple- 
mented. Educational Services was 
more strongly integrated iii'to the 
program. In the first year, clients 
were referred to Educational Ser- 
vices at the discretion of their 
counselor. In phase two MYEC 
cl ients were automatically referred 
to Educational Services if they 
showed 1 ow reading or math skil Is . 
A written recommendation of instruc- 
tional objectives was prepared and 
discussed with the client. Referred 
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clients were not required to receive 
training from Educational Services, 
but it was emphasized to them that 
their chances of success in the 
employment market were remote with- 
out basic skills remediation. 

The sv;itch to a structured work- 
shop format, and to more formal uti- 
lization of Educational Services 
changed the MYEC program from one of 
providing ad hoc one-on-one counsel- 
ing in vaguely defined job readiness 
skil Is, to one in which all cl ients 
were provided with training in much 
more tightly defined skills. We can 
therefore be much more confident 
about the job readiness skills of 
the 1982 graduates of the MEC pro- 
gram than we can about the 1981 
graduates. 

Problems . in implementing MYEC . 
The MYEC program suffered from man- 
agement difficulties and strained 
labor relations from its beginning. 
It was a large project. The project 
director was responsible for hiring 
staff and overseeing the design and 
impl ementation of three complex 
interventions. The project had a 
full-time staff of about 20 persons 
and involved negotiations with the 
evaluator , OJJDP, the Milwaukee 
Public Schools, the juvenile court, 
local CBOs, upper level JVS manage- 
ment, and the JVS employees' union. 
The newly hired project director had 
little experience in management, and 
his personal inclinations ran toward 
providing direct services to 
clients, not to running a large pro- 
j ect . The proj ect director did not 
have the management experience , or 
the exposure to program development 
to adequately oversee this complex 
task. The proj ect director adopted 
a laissez-faire management style , 
which was in keeping with his pre- 
ference for service rather than man- 
agement. 

The project director appeared 



often to avoid rather than confront 
problems. Tliree events involving 
the evaluation illustrate this avoi- 
dance of confrontation in uncomfor- 
table situations. Tlie project had 
promised to secure court contact 
data on treatment and control 
youths. Despite the obvious inter- 
est of both the evaluators and OJJDP 
in these important data in a del in- 
quency prevention project, and des- 
pite assistance in preparing to per- 
form this task, the court data v;ere 
not collected. Although the request 
was made in September , 1981 , 15 
months before the project was termi- 
nated, the project director did not 
negotiate with the clerk of court to 
^''get the required data. 

As a se cond exampl e , the proj ect 
director did not inform the evalua- 
tion staff of an upcoming strike in 
the spring of 1982. The result was 
that a follow-up of clients ground 
to an inexplicable (to us) Kal'tT and 
we were in the dark as to how to 
provide assistance. We speculated 
that no mention v/as made of the 
labor disputes in order to avoid the 
appearance of difficulties. 

A third exampl e invol ved the 
development of a strategy to pay 
experimental control subjects for 
their participation in a MYEC fol- 
low-up. Tlie project director had 
led evaluation staff to believe that 
the system for payment was complete 
and approved. It was not until the 
evaluation was over a month overdue 
in starting that the director 
informed evaluation staff that, in 
fact, he had not gotten the agree- 
ment of his comptroller to pay con- 
trol subj ects. 

A poor relationship between JVS 
and its employees' union made man- 
agement of the proj ect dif f icul t. 
The director, who had not been pre- 
viously associated with JVS v/as 
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immediately faced with the task of 
hiring staff in an atmosphere of 
distrust between JVS management and 
its employees. In the two years of 
project operation, there was one 
union strike, and a second was nar- 
rowly averted. 

Union regulations severely con- 
strained the director's efforts to 
select qualified staff. Immediately 
prior to pr oj ect start-up , JVS had 
been forced to lay off many of its 
employees. When the Alternative 
Education Project began, the project 
director was forced to hire staff 
from a pool of union employees avai- 
lable for recall . He had to hire 
from the list based on seniority, 
with little attention to qualifica- 
tions. 

The problem of forced hiring 
based on union seniority resulted in 
particularly acute problems in 
staffing ^^YEC's counseling unit. 
The union felt that virtually anyone 
was qualified to be a counselor, and 
pressed the issue. The result was 
that the backgrounds of the counse- 
lors who were charged with imple- 
menting MYEC s pr imary intervention 
were in diverse fields, but not in 
counseling. Two of the persons who 
served as counselors did not attend 
college; others had B.A. 's in 
sociology , business, and physical 
education. Another counselor had a 
B.A. in education, but had never 
taught, and her prior work experi- 
ence was as a buyer for department 
stores. The original supervisor of 
the counseling unit did hold a Mas- 
ters of Education with a major in 
Counseling, but she was replaced by 
a counselor who held a B.A. in Cri- 
minal Justice. Although degrees do 
not guarantee successful counsel ing , 
the lack of counseling training in 
the MEC counseling staff undermines 
conf idence that sound counsel ing 
practices were implemented. 



Staff of the Employer Services 
and Educational Services unit appear 
to have had more preparation for 
their work. Most of the Employer 
Services staff had backgrounds in 
business , or had previously worked 
as job developers on other JVS pr o- 
j ect s. All Educational Services 
teachers were certified teachers : 
the requirement that they be certi- 
fied al lowed the supervisor of Edu- 
cational Services to hire outside 
the JVS union. 

High turnover and strikes by JVS 
union (which included nearly al 1 
MYEC employees) created other pr ob- 
lems for MYEC. By the end of the 
proj ect , in November, 198.2, 11 per- 
sons had terminated out of the 26 
different persons who ever worked 
full-time on the project. Two 
extensive staff re-organizations 
were carried out by the project 
director in the space of a year and 
a half. Tlie average length of stay 
on the two-year project was 13.6 
months (and this does not include 
several JVS personnel who spent con- 
siderable time on the project, but 
were not paid out of grant funds). 
Project staff did not get along well 
with one another • As al ready men- 
tioned, there was one strike in 
October of 1981 , and a second was 
avoided in April 1982 in a "final 
hour" settlement: MYEC staff had 
f ul ly expected the employees to 
strike. MYEC sought help in staff 
relations from the technical assis- 
tance contractor (who suggested that 
even more assistance in this area 
might be helpful). 

Poor staff relations were not 
confined to problems between line 
and supervisory staff: the proj ect 
director and his supervisors in JVS 
did not work well together. The 
project director was hired from out- 
side the organization. He had pre- 
viously worked in a local CBO. The 
director disliked the JVS organiza- 
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tion, and never considered himself a 
part of it; he felt he was hired to 
run MYEC, and that MYEC should be an 
independent entity from the organi- 
zation at large. In turn, JVS did 
not trust the project director to 
manage the proj ect , and placed him 
under the immediate supervision of 
the Director of Youth Services. At 
one point during the summer we were 
told that the project director had 
been or v;as about to be dismissed; 
shortly thereafter we learned that 
the project director had filed suit 
against JVS for discrimination in 
pay. The project director was still 
holding that job in November when 
the project ended. 

Another problem for the project 
was the number of cl ients being 
served by the project. The pro- 
ject's proposal promised to serve a 
large number of persons, and the 
sponsor pressed the proj ect to meet 
its client load target. The program 
was begun before developing strong 
referral links to local CBO's, 
Juvenil e Court , Milwaukee Publ ic 
Schools, or the county Department of 
Social Services. Tlie result was 
that despite a full-time staff aver- 
aging over 17 persons , the program 
served fewer than 350 cl ients its 
first year of operation, and most of 
these came in a burst at the end of 
the year; by December, 1981, only 79 
youth had completed the entire MYEC 
job readiness program. 

In summary, MYEC's implementation 
dif f icul ties fall into four catego- 
ries : (a ) the proj ect director had 
1 ittle management experience , and 
his personal management style was 
incompatible with the demands of a 
large, multifacited project, (b) the 
proj ect director did not get along 
with his supervisors, and did not 
consider himself a real part of the 
JVS organization, (c) union hiring 
demands resulted in placing workers 
in the counsel ing unit who had 1 it- 



tie training or experience to work 
as counselors, (d) poor JVS manage- 
ment-labor relations put the project 
in continual conf 1 ict with its own 
staff, - even to the point of a 
strike, and (e) the project was 
begun v/ithout first establishing an 
extensive client referral network. 
Thus the project careened from one 
crisis situation to the next, with 
little time for planning, or 
thoughtful program development. 

The Job Score Class 

The Job Score Classes were an 
offshoot of the MYEC job counseling 
program, designed and directed by 
the supervisor of MYEC's educational 
services unit. They were an attempt 
to provide to students still in 
school the job readiness training 
available to dropout youth referred 
to MYEC. 

One of the prime mo tiv a it ions 
behind the instigation of the 
classes was the desire by OJJDP to 
see the JVS project more involved in 
"al ternative education" — the basis 
of the federal initiative. OJJDP 
did not regard MYEC, which was an 
al ternat ive to formal school ing as 
"alternative education." The Office 
insisted that the JVS project 
develop programs to serve youth that 
had not already dropped out of 
school , and the Job Score Cl asses 
were an attempt to meet this 
requirement. 

A tentative curriculum was 
designed in the summer and f al 1 of 
1981, and a contract between JVS and 
MPS providing for the implementation 
of the class in six area high 
schools was signed in October, 1981. 
Classes actually began in four of 
the high school s in January , 1982. 
Four more classes were held in the 
Fall of 1982. 



402 



JVS 



The Job Score class v/as a regular 
high school class that could be 
taken for credit toward the high 
school diploma in the Milwaukee 
Public Schools. The curriculum 
revolved around the MYEC job readi- 
ness skills. Topics included hov7 to 
fill out a job application, how to 
interview for a job, what to expect 
on the job, how to locate jobs, and 
how to get along with employers and 
f el 1 ow empl oy e e s . Al s o in cl uded in 
the curriculum was instruction in 
basic life survival skil 1 s such as 
how to balance a checkbook, shop for 
insurance , and figure interest 
charges. 

Several weeks into the semester 
(optimally, around six weeks) those 
students that were judged by the 
class teacher to be "job ready" 
(using the same criteria as clients 
in the MYEC program, above) were 
referred to ITiEC for job referral. 
Those students that MYEC also judged 
to be job ready were then referred 
to part time jobs. Students that 
landed jobs worked half time, and 
attended school half time. They 
received academic credit toward 
their diploma for the time they 
spent working . The program design 
called for students to maintain 
their academic standing, but it is 
not known if this requirement was 
adhered to . Student s who did not 
find jobs continued in the regular 
curriculum. 

The curriculum package as devel- 
oped by the supervisor of Education 
Services appears promising. She 
wrote detailed day-by-day lesson 
plans with clearly specified 
instructional obj ectives , and 
included methods of assessing suc- 
cess. All of the MYEC job readiness 
skills were covered and reinforced. 
The curriculum appears to be highly 
relevant to students, and was favo- 
rably viewed by the four teachers 
and principals involved in the ini- 
tial application of the course. 



The Return Center 

The Return Center v/as the focus 
of the original grant proposal to 
OJJDP. It was envisioned as one of 
a number of centers that would be 
set up statewide as a cooperative 
venture of all Wisconsin agencies 
responsible for providing social 
services to youth. 

These service delivery centers 
were supposed to ,be centralized 
diagnostic, referral , •and record- 
keeping centers . Their purpose was 
to serve as sites where youth with 
problems could be assessed and then 
referred to an appropriate service 
'delivery provider in the area. Cen- 
tral ized record-keeping would pro- 
vide a method of follow-up and eval- 
uation, and would allow agencies to 
better plan for the needs of youth. 
JVS would focus on developing a 
Return Center that would serve the 
needs of Milwaukee area youth who 
had dropped out of school, or who 
were contemplating dropping out, but 
who sought some form of structured 
education. The Return Center would 
assess their skills and interests, 
and then refer them to agencies 
(including the Milwaukee Public 
School system) that provided appro- 
priate intervention. 

It is perhaps symptomatic of the 
difficulties JVS had in implementing 
its project that, despite the intent 
of the or ig inal gr ant pr opo sal to 
focus on the Return Center concept, 
the JVS-sponsored Return Center did 
not begin operation until January of 
1982 — more than a year after the 
project was funded. 

Three maj or pr obi ems contributed 
to the slow implementation of the 
Return Center: 

1. The project director believed 
that he had been hired primarily to 
oversee the development and manage- 
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merit of the MYEC job counseling 
component of the program. It was 
his belief that the Return Center 
was primarily the responsibility of 
upper level JVS staff, and indeed, 
at the March 1981 planning confer- 
ence in V/ashington, D.C. a staff 
person not funded by the project 
accepted responsibil ity for negoti- 
ating the Return Center contract 
with IIPS . At the conference the 
proj ect director was assigned res- 
ponsibil ity for developing contacts 
with MPS in-school personnel so that 
referral sites would be available to 
accept clients from the Return Cen- 
ter. 

The project director had little 
experience in the negotiation of 
contracts and agreements, and was 
not happy with working with the Mil- 
waukee Public Schools. In fact, the 
project director (and MYEC staff in 
general) were reluctant to refer 
clients in the MYEC program to MPS. 
Also, the implementation of the MYEC 
component itself was not progressing 
rapidly, largely due to labor diffi- 
culties. The result of all of these 
forces was that the project director 
assigned top priority for his atten- 
tion to the MTEC job training pro- 
gram, and put the Return Center on 
the back burner. Negotiations for 
the establishment of the center were 
left to JVS staff outside of MYEC. 
The project director felt that if 
JVS upper management thought the 
Return Center was so important, then 
they could oversee it. In short, 
the proj ect director f el t that he 
was director of the MYEC portion of 
the project, and that the Return 
Center was an additional (relatively 
unimportant ) duty that took time 
away from what he saw as his central 
task. 

2. The second major reason for 
the slow impl ementation of the 
Return Center was that JVS and MPS 
had poor inter-organizational rela- 



t ions . JVS feared that MPS coul d 
not be held accountable for the 
funds, and would insist on control- 
ling the design and staffing of the 
Center. MPS suspected that JVS 
woul d real ly 1 ike to keep the funds 
"in-house" and would be reluctant to 
refer "JVS" youth to the Center, and 
back to MPS. Since neither organi- 
zation trusted the other, negotia- 
tions to set up a Return Center, 
staffed by MPS personnel, but paid 
for out of JVS/OJJDP funds, became 
long and extremely complicated. 

3. The third reason for slow 
implementation of the Return Center 
was that the negotiations for the 
establishment of the Return Center 
were tied to those for developing 
and implementing MYEC-sponsored Job 
Score classes in six Milwaukee 
public schools — for which curriculum 
did not exist in 1981. Educational 
Services personnel within MYEC were 
assigned responsibil ity~^foF design- 
ing the curriculum, and did so, but 
in the meantime, negotiations for 
the Return Center could progress no 
more rapidly than those for the Job 
Score classes (and vice-versa). 

The result of the three major 
obstacles to the implementation of 
the Return Center was that a draft 
of the contract with MPS was not 
completed until September, 1981. By 
the time a site was found and staff 
were assigned, it was January, 1982. 

To summarize the primary causes 
of the si ow impl ementation of the 
Return Center: (a) the JVS project 
director concentrated his energies 
on developing the MYEC component 
while paying little attention to the 
Return Center (and felt justified in 
so doing since other JVS personel 
took responsibil ity for negotiating 
the Return Center contract with 
MPS ) ; (b) the contract negotiations 
progressed slowly due to poor rela- 
tions between JVS and MPS ; and 
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(c) the Return Center negotiations 
were tied to negotiations for estab- 
lishing Job Score classes in the 
Milv;aukee public high schools. 

As finally implemented, the 
Return Center staff consisted of a 
program coordinator, a psychologist, 
a social worker , a vocational 
counselor, three teachers and a sec- 
retary. 

The intervention had three (some- 
times four) components: 

1. A reentry conference was held 
in which the youth (sometimes with 
his or her parents) met with Return 
Center staff to initially assess and 
interest the youth in participating 
in school activities. 

2. An assessment phase followed 
in which formal assessment of the 
youth^s basic academic level, intel- 
ligence, and vocational interests 
was made. 

3. A referral conference was 
held (sometimes with the youth's 
parents in attendance), in which the 
child was referred to either an MPS 
or Community-Based Organization 
(CBO) for instruction, or was re- 
ferred in-house to receive individu- 
alized instruction at the Return 
Center itself. 

4. Youth remaining at the Return 
Center for instruction received 
individual tutoring , usual ly 
directed at compl eting the G. E.D. 
Most youths who stayed at the Center 
did so because they wished to return 
to an MPS site, and could not do so 
immediately because it was too late 
in the academic semester. They were 
placed in MPS programming at the 
beginning of the next semester. 



Evaluation Results 

Results for the Milv7aukee Youth 
Employment Center 

Two types of research were per- 
formed to examine the effects of the 
MYEC component of this project. 
First , straightforward analyses of 
the experimental results were per- 
formed to describe the effects of 
the proj ect . Then, more compl ex 
analyses were performed to unearth 
as much information -as possibl e 
about ,the relative effectiveness of 
various elements of the overall MYEC 
activities. 

Primary analyses of MYEC effects . 
JVS successful ly implemented a true 
experimental evaluation of the MYEC 
job finding intervention. Beginning 
in August, 1981 and continuing until 
February, 1982, 252 intake youth were 
randomly assigned to treatment , and 
154 to control . Randomization was 
accpmpl ished on-site by using an 
intake bl ank on which 62.5% of the 
lines contained a "treatment" desig- 
nation and 37.5% of the lines con- 
tained a "control" designation (a 
five-to-three ratio of treatment to 
control clients). These blanks were 
prepared by the national evaluator 
using computer generated pseudo-ran- 
dom numbers. 

Applicants^ names were entered on 
this 1 ist in the order they pre- 
sented themselves at MYEC. Control 
youth were referred to other pro- 
grams for youth in the Milwaukee 
area. At intake, both treatment and 
control youths were administered a 
pretest that measured several of the 
objectives of the MYEC program. 

Seven months after intake, treat- 
ment and control youths were con- 
tacted by MYEC and JVS outreach 
staff. Each was individually admin- 
istered a questionnaire that ass- 
essed several personal char act eris- 
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tics, past and current employment 
status, and involvement in educa- 
tional activities. Control youth 
were paid $10.00 for their partici- 
pation. A total of 203 treatment 
youth were surveyed, for a 81% fol- 
Icw-up rate; 118 of the control 
youth were administered the questi- 
onnaire for a 77% rate. The two 
follow-up rates do not differ signi- 
ficantly (p = .41 by the chi-square 
test). 

The pre- and post-test measures 
of personal characteristics were 
subjected to principal factor analy- 
sis with iterated communal ities. 
Three factors had eigenvalues 
greater than 1.0 and were subjected 
to VARIMAX rotation. Separate ana- 
lyses of the pretest and post-test 
items yielded similar factors. The 
three factors were labeled Psycholo- 
gical Health, Interpersonal Compe- 
tency, and Rebellious Autonomy. 

Scales were constructed from the 
items loading most highly on each of 
the three factors; a particular item 
was only included on the scale on 
which it had its greatest loading. 
Although the three factors were 
similar for the two testing periods, 
there were differences in a few 
items . Various combinations of 
items were subj ected to internal 
consistency item analysis to develop 
a set of items that worked well for 
each of the three scales at both 
times. Appendix A gives the item 
content for the three psychosocial 
measures developed through these 
procedures. Psychological Heal th 
had a rel iabil ity coefficient 
(Chronbach's alpha) of .61 at pre- 
test, and .59 at post-test, and a 
test-retest correlation of .40. 
Rebel lious Autonomy had alpha coef- 
•f ici ent s of .49 at bo th pr e- and 
post-test and a test-retest rel i- 
ability of .39. The third scale. 
Interpersonal Competency had an 
alpha of .45 at pretest, and .46 at 



post-test. This scale shows lower 
test-retest rel iabil ity than the 
other two scales ( .19) . 

Table 1 gives the results of 
post-randomization checks on the 
equivalency of the MYEC treatment 
and control groups. Chi-square 
tests on the sex of the subjects and 
the percent who were of minority 
ethnicity (mostly Black and His- 
panic ) , show no differences between 
the groups on these measures. Ana- 
lyses of variance indicate that the 
groups V7ere similar in age, and in 
the time of intake (there is a 
trend, p = .10, toward the treatment 
youth being slightly older than the 
controls, but the absolute size of 
the difference is very small — see 
Table 1). The groups did not differ 
on Lhe time between intake and fol- 
low-up. 

Analyses of the pretest psychoso- 
cial measures show no differences on 
Psychological Health, or Interper- 
sonal Competency. Tlie control group 
reported significantly greater 
Rebellious Autonomy (p=,006) than 
the treatment group. This result 
means that this pretreatment differ- 
ence will have to be accounted for 
in any analysis of outcomes, parti- 
cularly on Rebellious Autonomy as 
measured at fol low-up . Given the 
procedure used to assign youths to 
treatment and control groups , and 
the quite close equivalency of the 
groups, we are confident that a true 
experiment was implemented. 

Changes in the psychosocial mea- 
sures were assessed by subtracting 
the follow-up score from the pretest 
score. Mean changes, and results of 
analyses of variance carried out on 
the scores may be found in Table 2. 
No differences were statistically 
significant. The treatment and con- 
trol youth showed no differential 
change in Psychological Health, 
Interpersonal Competency, or Rebel- 
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lious Autonomy. An additional 
attitudinal measure not available at 
pretest was developed for the post- 
test. It was called Attitude Toward 
Education. Tliis v;as a scale com- 
posed of five items designed to mea-. 
sure the degree to which the respon- 
dent perceived formal education as 
useful and rel evant to his or her 
1 if e . This measure was developed 
because Educational Services staff 
felt that one of Lneir primary 
objectives was convincing youths 
that education was indeed relevant 
to their lives and livel ihood. 
Tabl e 2 shows that there v/ere no 
differences between the experimental 
and control groups on this scale. 

The analyses of the measures of 
MYEC's intermediate objectives gives 
no reason to bel ieve that they were 
met. Nothing in these results sug- 
gests that any additional job exper- 
ience provided to cl ient youth by 
the MYEC program resulted in any 
positive psychosocial change. 

More critical for the MYEC evalu- 
ation is an analysis of the current 
and past employment and educational 
status of MYEC clients and their 
controls. Table 3 gives the results 
of chi-square analyses comparing 
treatment and control youth on three 
measures of program success: cur- 
rent employment status, current edu- 
cational status, and a combined edu- 
cational and employment status 
measure. For this composite mea- 
sure, a "positive outcome" was 
regarded as a report of being in 
school or employed. These outcome 
measures were derived from a ques- 
tion asking *Vhat are you doing 
mostly?" The respondent could only 
check one of several categories of 
educational involvement or employ- 
ment. Analyses indicate that the 
MYEC clients are no more likely to 
report themselves to be currently 
working on a job than are the con- 
trol youth. Approximately 80% of 



both groups v;ere not working at 
follow-up. There is a tendency for 
the control group to report more 
involvement in school (either re-en- 
rolled in high school, or attending 
a G.E.D. program). This tendency, 
when taken together with the very 
small difference in work status, 
produces a trend (p = . 07 ) for a 
higher percentage of control s to 
report a "favorable outcome" (either 
working or iT^ school) than treatment 
youth. 

The difference in reported school 
attendance is dif f icul t to inter- 
pret. Project staff have repeatedly 
noted that at intake clients invari- 
ably report that they are working on 
their G.E.D. , even if they have 
never enrolled, or have never 
attended a G.E.D. class. Many young 
dropouts believe that they will 
graduate, "some day." MYEC clients 
are faced with the fact that if they 
truly want to earn their G.E.D., the 
training is availabl e at MYEC 
through Educational Services. It is 
therefore more difficult for MYEC 
clients to claim that they are work- 
ing on a G.E.D. , if they are not , 
than for the control youth to make 
such a claim. Altho ugh we have no 
way of verifying the respondents'", 
reports it is doubtful that the 
experimental and control groups dif- 
fered in participation in a G.E.D. 
program. j 

Further information on employment 
and educational status can be found 
in Table 4. This tabulates / the 
results of chi-squares comparing 
responses to various categories 
availabl e in response to a question 
asking "What are you doing now?" 
Allowing mul tiple responses enabled 
students who were working part-time 
and going to school to record both 
activities. 

As the table shows, the results 
of the analysis of the questionnaire 
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item measure mirror those for the 
previous item. About equal percen- 
tages of treatment and control 
youths report they are currently 
working for pay, but more control 
than treatment youths report that 
they are attending a G. E. D. program 
and more report that they are 
enrolled and attending high school. 
More treatment than controls acknow- 
1 edge that they are unempl oy ed 
(v.'hich also connotes not attending 
school). A higher proportion of the 
control youths also report that they 
are "On temporary layoff from work." 

Tables 3 and 4 -agree in implying 
that (a) the MYEC clients are no 
more 1 ikely to be empl oyed at fol- 
low-up than are the control s (and 
very few of either are employed) , 
and (b) a higher proportion of con- 
trol group members report themselves 
to be re-involved in some kind of 
formal education. Of particular 
interest is the difference in the 
proportion of respondents from the 
two groups who report they are 
"Enrolled in and attending high 
school." We cannot be sure that the 
control s were simply "yea saying" 
more than the treatment stu- 
dents — although the lack of differ- 
ences in responding to work-related 
questions would argue against that 
hypothesis. But both Tables 3 and 4 
suggest that participation in MYEC 
may have discouraged students from 
returning to formal school ing. The 
results offer no evidence that MYEC 
was successful in aiding cl ients to 
find more stable employment than 
they could have found on their own. 

Other follow-up questions con- 
cerned hourly wages, current hours 
worked per week, and hours worked 
f ul 1 and part-time "in the last six 
months. " This insured that al 1 
repondents woul d be reporting only 
on employment experience acquired 
since MYEC intake. Table 5 gives 
the results for analyses of variance 
for the wage and hour measures. 



None of the nine measures reveal 
any differences between the treat- 
ment and control groups. The groups 
do not differ on the current num.ber 
of hours working per week, current 
hourl y wage s , current we ekly wage s , 
or total wages earned in the last 
six months. Tlie groups are also 
similar in the the number of weeks 
that they had worked full time and 
part time. Finally, a logarithmic 
transformation of the current hourly 
wages and total wages earned (a 
transformation commonly carried out 
on wage data to decrease the influ- 
ence of extreme values in what is 
usually a markedly skewed distribu- 
tion) did not produce any evidence 
of differences between the treatment 
and control groups. 

There is no evidence in these 
results that MYEC improved the 
employment situation of its clients. 
They are no more likely to be 
employed at follow-upT, ~to have 
worked more in the recent past, or 
to be earning or have earned any 
more money . There is therefore no 
evidence in these results that the 
MYEC program met its primary goal of 
increasing the employment opportuni- 
ties of its clients. 

The MYEC pr oj ect was funded pri- 
marily as a delinquency prevention 
effort. Employed youth were theor- 
ized to have less of a need to com- 
mit crime out of a need for money , 
for status , or for .excitement ; 
Because the treatment group did not 
differ from the control group in 
employment status, one would not 
expect (on the basis of this theory) 
delinquency to be differentially 
affected. Yet , personal counsel ing 
was a largu part of the MYEC pro- 
gram, and might be expected to have 
some positive effect on del inquent 
behav ior independent of empl oyment 
effects. There is no evidence that 
it did. The final outcome listed on 
Table 5 shows results for a question 
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Oot^ cinit^t oiiily «s«f)i» the odo<« 
tMcy of « progrMi^i ihoorrticAl 
)^f«iiti» or lu roUtlon to Iti 
ttioor|if« tmloii 000 Hoovii that tho 
K^lfM %oi rottiOMbSy vttU ixnplt** 
MM (hit Toblo 6 offors »trong tvl- 
i«oco Itiit tho KfSC job counttling 
firogrtii vat oot iskpX o««nt«d with 
ooottgh fUfUty to rooch utrong eon- 
olutiofti obout its potontittl offi- 
Cicy* 

Tiblo h tlvoft limpU dosc«*iptivo 
ittti tlici Of four impl mtnta t ion 
Mttioroit tilt nottbor of counsel Ing 
cofttitttf tho luncihor of i^ucitional 
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Job* to vhich tho client vas re** 
forrt4« Md the ntimber of Jobi that 
tho elieot voi actual ly placed in by 
MYSC« 9toat inproitivo is tho Uat 
col %mak in Tabl o 6 • It ihova the 
percootagea of clienta receiving 
jBjgmt ^ that particular part of the 
ifttorvootion* Tho bottom line» of 
coorie» la Job placenenta^placing 
youth io jobt ia what KYEC was all 
oboot* Vet, nt of the KYEC clienta 
^twoon Aiiguat 1981, and February, 
Ittl were oot placed In a job. In 
other worda, mly a little over a 
^oarter of KfSC clients received 
KYCC^a main service* Only half were 
over evoo raf*ri'#d to a job* For 
each of tho four iKplementation mea-* 
attrea» the sodal (moat common) 
Msooot of service provided to a par* 
ti€olar cltesit was aero* 

A^roatly the KYEC project was a 
Job placement project that did not 
{ilaco many yootha In Joba* In other 
%rorda„ tho Job counseling/placement 
program waa not Impl emented as 
intended* Vo cannot clearly address 
tho question of the theoretical ade-* 
qoacy of tho MTEC program, or its 
relation to Its theory (the first 
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and second possible reasons for pro- 
grnm failure listed above) because 
the evalu ated MYEC program vas not 
the .intonded MYEC program. As to 
whether the evaluation design was 
senbitive to possible program 
effects, a well implemented true 
experiment is a strong design (Cook 
and Campbell, 1979). The high res- 
ponse rate from a large number of 
OKper iitental tx'eatment and control 
participants implies that this 
aspect of the experiment was faith- 
ful ly impl emented . Al though ques- 
tions could be raised about the mea- 
sures examined, the job- and 
education-related measures appear 
straightforward and highly relevant 
to the evaluation of this project. 

A further question might be, "If 
MYEC had systematically implemented 
the program, would it have been able 
to place more of its clients in 
jobs?" In order to investigate this 
a question, a series of regression 
analyses were undertaken. Statisti- 
cal models were built to account for 
(in separate analyses) individual 
variation in the five pr imary 
employment measures, an indicator of 
enrollment in school, and self-re- 
ported police contacts. 

Two regression models were 
employed to investigate effects of 
differential exposure to MYEC treat- 
ment. In the first model, for each 
MYEC client a single indicator of 
MYEC treatment intensity was entered 
which reflected the total amount of 
MYEC treatment received. In the 
second model , each separate compo- 
nent of the MYEC program was indivi- 
dual ly examined . In both model s , a 
stepwise procedure was employed in 
which background and personal ity 
measures at intake to MYEC were 
entered first ; sex, ethnicity, age , 
month of enr ol Iment in th e pr ogr am , 
Psychological Heal th at pretest. 
Interpersonal Competency at pretest. 
Rebellious Autonomy at pretest, and 
basic skills prof iciency were 



allowed to enter at this stage if 
they contributed to the explanation 
of the outcome of interest. 

In the f irst set of regression 
analyses, MYEC treatment effects 
were studied by entering at step two 
an overall index of MYEC treatment 
received. Any additional variance 
in the individual employment, 
schooling, and delinquency scores 
that was captured by this step in 
the analyses might be attributable 
to MYEC treatment, net" of measured 
backgr.ound characteristics. 

The MYEC treatment intensity 
score was constructed by dichotomiz- 
ing the four MYEC implementation 
measures at the median. Each MYEC 
client received a score of 1 on each 
measure for which he or she received 
more than the median amount of 
treatment , and 0 if he or she 
received less than the median amount 
of treatment . These four scores 
were then summed to yield a single 
index of treatment intensity 
received. A client's treatment 
intensity score could range from 
zero to four. 

Table 7 reports the results of 
the MYEC treatment intensity regres- 
sion analyses on the primary employ- 
ment variables: hours currently 
working per week, weeks worked 
full-time in the last six months, 
weeks worked full-time in the last 
six months , log-transformed current 
wages, and log-transformed total 
earnings in the last six months. 
Listed are the effects of a) those 
predictor variables entered at step 
one that significantly increased the 
variance explained , and b) the MYEC 
treatment intensity variable 
(labeled "MYEC participation"). The 
column headed "beta" shows the con- 
tribution of each variable, net of 
any previously entered var iabl es. 
The beta given for the MYEC partici- 
pation score is for the direct 
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effect of MYEC treatment, indepen- 
dent of all examined background mea- 
sures. That is, the beta represents 
the MYEC treatment's contribution to 
explanation that cannot be attri- 
buted to any other variable in the 
model. The third column headed 
"increment to R^" shows the propor- 
tion of variance in the criterion 
accounted for by adding that predic- 
tor to the model, net of previously 
entered predictors. Tlius, the "MYEC 
participation" entry gives the added 
variance in the predictor explaina- 
ble by increased MYEC participation, 
holding constant the background mea- 
sures. Finally, the last coluinn, 
headed "p for increment" provides a 
statistical test for the added con- 
tribution (incremental validity) of 
each successive variable. The p's 
for MYEC participation are of parti- 
cular interest — significant incre- 
ments in explained variance (marked 
by an asterisk) suggest that parti- 
cipation in the MYEC program proba- 
bly contributed to an increased 
score on the criterion measure 
investigated. 

Table 7 indicates that the MYEC 
intensity variable was a significant 
predictor of empl oyment status for 
four of the five employment depen- 
dent variables: receiving more MYEC 
treatment is associated with more 
hours per week currently worked, 
more part time work engaged in dur- 
ing the last six months, higher cur- 
rent wages earned, and greater total 
wages earned during the last six 
months. 

Table 7 also indicates that age 
and ethnicity are consistently asso- 
ciated with employment outcomes. 
Older youth show better employment 
and earnings than blacks . There is 
al so some ev idence that greater 
Interpersonal Competency as measured 
at pretest predicts less part-time 
employment , and, therefore , less 
total earnings over the last six 
months. 



Table 8 gives the results for 
regression analyses investigating 
the degree to which MYEC treatment 
participation can predict the psy- 
chosocial outcomes of Psychological 
Heal th , Interpersonal Competency , 
and Rebel 1 ious Autonomy as measured 
at post-test , re-enrollment in for- 
mal school ing, sel f-reported pol ice 
contacts , and Attitude Tov/ards Edu- 
cation as measured at post- test . 
Again, background variabl es were 
entered first, and the MYEC partici^ 
pat ion variable last the effect 
given , for the 1-fYEC variable under 
the heading "beta" indicates the 
direct effects of MYEC participa- 
tion, independent of any variables 
entered previously. 

Table 8 shows that greater MYEC 
treatment has no statistically sig- 
nificant net contribution to psycho- 
social development (although there 
are trends toward treatment being 
negatively associated with Rebelli- 
ous Autonomy, p=.08, and positively 
associated with Attitude Toward Edu- 
cation, p=.10). Table 8 also sug- 
gests that increased participation 
in MYEC activities is associated 
with an increased likel ihood of 
being re-enrolled in some form of 
formal schooling, i.e., MYEC clients 
who persisted in th e pr og r am were 
more likely to enroll in school . 
Note that this resul t differs from 
that found in analyses of the treat- 
ment and control comparisons. 
Resul ts of those analyses suggested 
that the MYEC program discouraged 
re-enrollment in school. This appa- 
rent discrepancy can be reconciled 
when it is recalled that most MYEC 
cl ients received 1 ittl e or no I'TYEC 
treatment. It would appear that for 
the majority of MYEC clients who 
received only superficial exposure 
to the program, MYEC discouraged 
them from becoming re involved in 
formal school ing — yet the program 
had the opposite effect on youth 
that persisted in the program and 
received the entire treatment. 
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Those regression aniilyses show 
that there is a mild negative pred- 
ictive relationship (beta=-.12, 
p=.13) between MYEC participation 
and self-report arrests ("picked up 
by the police during the last six 
months"), i.e., there is weak evi- 
dence in these data of a direct 
effect of MYEC participation upon 
the reduction of del inquency . This 
result is si ightly muddled in that 
police contacts are measured somew- 
hat concurrently with the MYEC vari- 
able — the youth are reporting on 
their last six months experience , 
part of which would have been spent 
in >rYEC. Some clarity is lent to 
this analysis by the fact that most 
clients were in MYEC for a short 
period of time, and even long-term 
cl ients woul d seldom have been in 
the program for a month. Since the 
average fol low-up period was about 
seven months, the majority of MYEC 
participation would have occurred 
prior to the period for reporting 
arrests. There is some legitimacy, 
therefore, in believing that the 
MYEC effect found (such as it is) is 
a true "predictive" relationship; 
the longer a youth persisted in MYEC 
treatment the less 1 ikely he or she 
was later to report being picked up 
by the police. 

Table 8 also indicates that the 
strongest predictor of future psy- 
chosocial development is present 
psychosocial development — personal i- 
ty is fairly stable across the time 
period covered by this study. Most 
of the variance in Psychological 
Health, Interpersonal Competency, 
and Rebellious Autonomy at post-test 
is accounted for by the same person- 
ality variables measured at pretest. 

To summarize the effects of 
increased exposure to ^^YEC treat- 
ment. Tables 7 and 8 show positive 
rel ationships between the amount of 
MYEC treatment on the one hand and 
employment and formal schooling sta- 



tus on the other, and some 
suggestion of a weak effect on del- 
inquency. 

In order to more closely examine 
the MYEC effects, a second set of 
regression analyses were carried 
out. In this set of analyses indi- 
^ vidual measures of each of the four 
^^YEC treatment variables were used 
to predict employment status (Table 
9) and psychosocial , school ing, and 
delinquency status (Table 10). In 
these analyses , the individual MYEC 
treatment indices were substituted 
for the overall MYEC treatment par- 
ticipation score in the second step 
of the analyses, following the entry 
of the background variables. Counts 
of counseling contacts, educational 
services contacts, job referrals, 
and job placements were allowed to 
enter the statistical model in the 
order of the amount of variance each 
explained. In all cases, statisti- 
cally significant effects listed are 
for the direct contribution of the 
significant MYEC treatn^ent compo- 
nent (s) toward the explanation of 

variance of the outcome measure in 
9 

question.^ 

Table 9 shows that the number of 
educational services contacts and 
the number of job pi acements are 
generally associated with more favo- 
rable employment outcomes. Counsel- 
ing contacts show little relation- 
ship to employment outcomes, except 
for a si ight tendency for those who 
received more counsel ing to be less 
involved in in part-time employment 
(p=.09). This tendency contrasts 
with the result for number of educa- 
tional services contacts which show 
a moderately strong positive 



2. Effects for the other variables 
in the model — Lhe background varia- 
tion — are not given in Tables 9 or 
10 because they are identical to 
those given in Tables 7 and 8. 
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relationship with part-ti.ine employ- 
ment (p<.001). ^tYEC records indi- 
cate that most of the jobs in the 
MYEC job bank during the time of the 
evaluation v;ere part time jobs. 
These resul ts suggest that cl ients 
availing themselves of the help 
offered by Educational Services were 
more successful at obtaining work 
than were other clients. The nega- 
tive correlation between counseling 
contacts and part-time employment 
suggests either that the counsel ing 
staff were actively discouraging 
clients from seeking part time work, 
or that youths who received counsel- 
ing also experienced more difficulty 
in getting jobs. There was no 
organizational pol icy of either 
encouraging or discouraging part- 
time work, and different parts of 
the MYEC project may have treated 
their clients differently in their 
approach to part-time work. 

Overall, Table 9 provides no evi- 
dence that counseling in the coun- 
seling unit was successful in pre- 
paring cl ient s for empl oyment . 
Controlling for background varia- 
bles, there is no significant rela- 
tionship between counseling and 
employment . Recal 1 that counsel ing 
was considered to be the primary 
intervention: educational services 
contacts were supposed to be an 
additional treatment available for 
clients desiring upgrading of basic 
skills. The present analyses sug- 
gest, however, that training in 
basic skills had more effect then 
the prescribed counseling. 

Table 10 shows the effects of the 
individual treatment components on 
psychosocial development , re-enroll- 
ment in school, and self-reported 
arrests. Only the implementation 
measure of job placements explains 
any additional variance in the psy- 
chosocial measures at post-test over 
that explained by the pretest and 
background variables. Being placed 
in a job by MYEC is associated with 



greater Psychological Health, and 
with more positive Attitude Toward 
Education. 

The overall treatment intensity 
analysis indicated that increased 
MYEC treatment was associated with 
increased re-enrollment in formal 
schooling; Table 10 shows that this 
effect is largely the result of the 
efforts of educational services. It 
is the only individual treatment 
component that is a significant 
predictor of re-enrollment in 
school . 

Finally, Table 10 suggests a 
small positive effect for job place- 
■'ments on police contacts (p < .10). 
Clients who were placed in jobs were 
slightly less likely to report them- 
selves to have been picked up by the 
police. Thus, the majority of the 
trend toward decreased self-reported 
arrests associated v/ith persistence 
in MYEC (see Table 8) is attributa- 
ble to actually being placed in a 
job. This is important for a theor- 
etical analysis of the relationship 
between employment and del inquency 
(see Appendix B) . 

A final observation from this 
analysis of MYEC treatment component 
effects is that the counseling com- 
ponent is essentially unrelated to 
employment (although somewhat nega- 
tively related to part-time work), 
and is associated with significantly 
less Interpersonal Competency at 
follow-up, i.e., the few effects 
noted are in a direction opposite to 
program objectives. 

In summary, the results of the 
foregoing regression analyses sug- 
gest that: 

1 . More exposure to the MYEC 
treatment is associated with better 
empl oyment status at fol low-up , but 
the primary evaluation — the true 
e.hw ex-per-i-ment in-di-cates no 
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overall positive benefit for the 
MYEC program. It is difficult, even 
with the variety of pretest and 
background information available on 
the clients, to entirely control for 
pre-existing motivational differ- 
ences among MYEC participants; youth 
that stuck to the MYEC program, and 
received more treatment, might show 
enhanced employ abil ity simply 
because they are harder workers , or 
are more persistent in a job search. 

Nevertheless, because we do have 
a number of statistical control 
variables, the results suggest that 
had clients received more systematic 
and thorough exposure to the treat- 
ment, the program would have been 
successful . 

2. Receiving assistance through the 
Educational Services component and 
actually being placed in a job by 
MYEC is associated with some posi- 
tive outcomes. No evidence suggests 
that counseling, MYEC's primary 
intervention, was successful in aid- 
ing MYEC clients to find employment 
or to improve their psychosocial 
skills. In fact, to the extent that 
pre-existing client differences were 
successfully controlled for by pre- 
test measures, the results suggest 
that counseling produced a decrement 
in Interpersonal Competency. This 
result is not necessarily an indict- 
ment of employment counsel ing in 
general . Recal 1 that the MYEC 
counselors had little training or 
experience in counseling. 

3. The educational services compo- 
nent of the program appears to have 
contributed to the employment and 
educational status of its partici- 
pants. Again, to the degree that 
motivational differences are cont- 
rolled for by the pretest and back- 
ground measures, basic skills educa- 
tion in an alternative out-of-school 
format increases the employabil ity 
of dropout youth. Because educa- 



tional services offered in-house 
G.E.D. instruction, it also contri- 
buted to the rcinvolvement of MYEC 
clients in formal education. 

The positive educational services 
resul ts are important for the over- 
all evaluation of the Del inquency 
Prevention Through Alterative Educa- 
tion initiative ; this al ternative 
education intervention appears to 
result in some positive outcomes. 

4. Three other consistent pred- 
ictors, of employment status are the 
racial status of the clients, their 
age, and the month the youth was 
enrolled in the MYEC program. 
Whites are more likely to be 
employed; older youth have better 
employment outcomes; and the later 
in the year that a youth was refer- 
red to MYEC the less they worked 
full- or part-time in the following 
seven months. 

The results for race imply that 
discrimination in hiring exists in 
the city of Milwaukee. Even when 
control 1 ing for background differ- 
ences, including basic skills profi- 
ciency, there is still a residual 
net benefit to being white. In 
fact, in most analyses', the beta for 
racial status is larger than that 
for receiving MYEC treatment. 

One might argue that some motiva- 
tional difference accounts for the 
differential employability of whites 
and minorities. This hypothesis can 
be explored by looking at persis- 
tence in treatment. Table 11 shows 
the simple correlations between the 
number of MYEC component contacts 
and white/minority status. Minori- 
ties stayed in treatment longer as 
measured by the number of counseling 
and educational services contacts. 
There is no evidence here to suggest 
that minority youth were unmotivated 
to attend program sessions. 
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The lack of correlation between 
raincity status and job referrals in 
Table 11 indicates that MYEC person- 
nel referred proportionately as many 
minority as non-minority clients for 
jobs. Yet the significant correla- 
tion between job placement and 
white /minor ity status demonstrates 
that MYEC was unable to overcome the 
basic white/minority difference in 
employability . Although MYEC spent 
more time with minority clients, and 
referred both whites and minorities 
equally, to jobs, white youths were 
still more successful in securing 
employment. 

The findings of enhanced employ- 
ability with increased age, and 
decreased employability as the pro- 
gram progressed are also important 
results. The age of MYEC's service 
population was a continual source of 
contention both within the project 
itself, and between the project and 
the federal sponsor. MYEC personnel 
felt they had been saddled by JVS 
with an impossible task: finding 
employment for 16- and 17-year-ol d 
dropout mostly minority youth during 
a recession. Particularly disturb- 
ing to the staff was the constraint 
they felt in being required to serve 
only 16- and 17-year-old youth. 
They felt that they could have been 
much more effective serving older 
dropout youth. ^ 

The finding of poorer employment 
status as the evaluation progressed 
is difficult to interpret. One pos- 
sibility is that the general Milwau- 
kee employment situation may have 
been deteriorating over the course 
of the evaluation as the recession 



3. The Office for Juvenile Justice 
and Del inquency Prevention's legis- 
lation limits its purview to juven- 
iles. Because the project was spon- 
sored by OJJDP, it had to serve 
juveniles . 



deepened, and jobs may have become 
increasingly hard to come by. The 
effect is not due to any increase in 
time between entry and follow-up. 
The time lag, which was measured for 
most of the follow-up population, is 
uncorrelated with any of the outcome 
measures . 

Whatever is the source of the 
decrease in full time employment for 
the MYEC clients over the course of 
the evaluation, there is nothing in 
the results to suggest that the pro- 
gram improved during the course of 
the evaluation. CI ients served in 
the later months of the project were 
less 1 ikely to work full-time than 
were clients served in the project's 
early months. Project personnel 
argued that the recession made their 
task almost impossible; employment 
was so high in general, and in their 
target population especially, that 
there were simply no j obs available 
in which to place the clients. 
Research indicates that youth unem- 
ployment is yery sensitive to the 
state of the economy — a one percent 
increase in the official adult male 
unemployment rate translates into a 
5-6% increase in the number of teen- 
age mal e s looking for work (Cl ark 
and Summers, 1980). In fact, Clark 
and Summers (1980) argue that the 
foremost reason for high teenage 
unemployment is a lack of j obs , and 
that it is fruitless to set up 
"employabil ity" programs (such as 
MYEC) . Such programs (even when 
they work) may merely substitute one 
youth for another in the employment 
queue with no net positive result on 
unemployment in the target popula- 
tion. This problem would be parti- 
cularly acute during a recession as 
severe as the one that occurred dur- 
ing MYEC's operation. 

Additional observations on MYEC . 
As discussed earl ier , the MYEC pro- 
gram underwent substantial change s 
during the winter of 1981-82. 
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Emphasis v;as shifted from a one-on- 
one personal counseling orientation, 
to a structured series of defined 
v;orkshops accompanied by increased 
input from Educational Services. 
Since the evaluation covered the 
period of August 1981 to mid-Febru- 
ary 1982, the follow-up essentially 
evaluated the early version of the 
liYEC program that the program per- 
sonnel themselves had found unsatis- 
factory. 

A second experimental evaluation 
of the MYEC project had been begun. 
A job readiness pretest covering the 
skills thought necessary for employ- 
ment success had been developed by 
project staff. This instrument 
would have served not only as a 
experimental pretest, but as a check 
on the program's success in provid- 
ing the skills it deemed important. 
This second experiment was to begin 
in October, 1982, but was delayed 
several times. It had begun on 
November 30, 1982, but the discon- 
tinuation of the project by the Off- 
ice for Juvenile Justice and Delin- 
quency Prevention aborted the second 
evaluation. 

The loss of the second evaluation 
is unfortunate because of the major 
changes that the MYEC project had 
undergone. Accordingly, the evalua- 
tion presented here is an evaluation 
of the program in development , not 
as developed . We know that a MYEC 
program based on one-to-one counsel- 
ing was not successful ; we do not 
know if the project modifications 
produced a more successful set of 
interventions. 

Evaluation Results for the Job Score 
Class 

In the spring of 1982, the Job 
Score class was begun in four Mil- 
waukee high schools. Together with 
the project staff, we attempted to 
implement two successive evaluations 
of the Job Score Classes. A 



preliminary experiment was attempted 
in all four schools, but the designs 
broke . down in three of the four 
schools : control students were not 
pretested (or identified), and ran- 
domization was not actually imple- 
mented because randomization took 
place before students in the pool of 
eligibles were asked whether they 
wished to be in the cl ass ; many did 
not. In one of four schools random- 
ization did take place, but in that 
school the col 1 ect ion of fol low-up 
data at the end of the • semester was 
incomplete. Of the 21 students 
assigned to treatment and 18 to con- 
trol, complete data were only avai- 
lable on 12 (57%) of the treatment 
students and 7 (39%) of the control 
students. Results of such a follow- 
up are virtually uninterpretable. 
The data were analyzed nonethel ess, 
and the treatment and controls did 
not differ on any outcome measure : 
Internal Control, Self-esteem, 
Interpersonal Competency^ Rebel 1 ious 
Autonomy, Alienation, Attitude 
Toward Education, Employment Status, 
or Drop-out. 

Having learned from their first 
experience, the experimental evalua- 
tion of the Job Score class in four 
high schools in the fall of 1982 was 
much better implemented. Randomiza- 
tion was carried out properly, and 
pretests on personal ity and job 
readiness skills were administered 
to all treatment and control stu- 
dents. Unfortunately, the experi- 
ment was terminated along with the 
project when OJJDP decided not to 
continue funding . The Milwaukee 
Publ ic School s were unwil 1 ing to 
complete the follow-up without the 
funding provided by OJJDP. 

In short , we know littl e about 
the effectiveness of the Job Score 
class. This is particul arly unfor- 
tunate because an enormous amount of 
careful work went into the develop- 
ment of its curriculum, and because 
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of the interest the project ''s 
sponsor had shown in developing this 
aspect of the project. One of the 
goals of the Alternative Education 
Initiative was to develop innovative 
alternative educational approaches 
for youth; the Job Score curriculum 
seems to have been aimed at that 
goal . 

Evaluation Results for the Return 
Center 

The Return Center did not begin 
operation until January, 1982. At 
that time MYEC personnel were begin- 
ing the follow-up of the MYEC exper- 
imental treatment and control sub- 
j ects . This follow-up was not 
progressing well, and we recommended 
that MYEC focus its energies on com- 
pleting the evaluation of the job 
counseling program aleady in pro- 
gress. One good evaluation was 
judged to be more worthwhile than 
two bad ones. In addition, since 
the Return Center was * a brand new 
intervention, we felt that it should 
be allowed to operate for some time 
and work the kinks out before being 
subjected to formal evaluation. 
Thus, no evaluation for the Return 
Center was designed for the spring 
of 1982. 

During the August, 1982 confer- 
ence , an evaluation of the Return 
Center was designed for implementa- 
tion in the fall of 1982. The 
design involved tracking the first 
125 referrals to the Center for both 
the first and second semesters of 
1982-83 to see how many actually did 
re-enroll in some kind of schooling, 
and what kind of progress they had 
made by the end of that semester. A 
comparison would be made between the 
success of youth referred to MPS 
alternative education sites, and 
youth . referred to CBO-sponsored 
training (mostly G. E. D. instruc- 
tion) . Since referral was largely 
based on geography — youth were 



referred to a program close to where 
they live — the de sign provided a 
provocative comparison of "official" 
alternative education versus CBO- 
sponsored alternative educationi — — 

The ev al ua t i on wa s ne v e r com- 
pleted. The evaluation began well 
and was progressing nicely when the 
proj ect 's funding was discontinued. 
Pretest and background information 
on the first 125 referral s had been 
col 1 ect ed, and fol low-up forms had 
been designed and approved by JVS, 
MPS, and the CBO referral sites. 
All information was being provided 
as requested. Because of the pro- 
^■ject's termination, we know nothing 
about the efficacy of the Return 
Center. 

Recommendat ions 

The evaluation of the JVS Alter- 
native ~Education"Proje'ct^^and" our 
interpretation of it , lead to the 
following recommendations: 

1. The Job Score class appears 
to be a carefully developed curricu- 
lar innovation worthy of dissemina- 
tion and testing. Procedures should 
be established to examine its effec- 
tiveness. 

2. Delinquency prevention pro- 
j ects involving individual counsel- 
ing should be staffed with profes- 
sionally trained counselors* Such 
counselors should, at a minimum, 
meet the guidelines or standards for 
counselors as laid out by the Ameri- 
can Personnel and Guidance Associa- 
tion, the American Psychological 
Association, the Association of Mar- 
riage and Family Counselors, or the 
laws of the state within which the 
proj ect operates. Paraprof essional s 
providing counseling services should 
be closely supervised by profession- 
al s . Counsel ing interventions 
should be rigorously evaluated. 
Funding agencies should require that 
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such minimum standards be adhered to 
by projects they sponsor. 

3. Experimental research, should 
be conducted on the consequences of 
the employment of dropout and poten- 
tial dropout youth. We should not 
have to rely on correlational meth- 
ods to answer policy questions that 
would be better answered experimen- 
tally. 

4. Experimental research should 
be conducted on the outcomes of 
returning dropout youth to school. 
We have long assumed that "the more 
education, the better." Some 
results presented here question this 
assumption. Under what conditions 
is it beneficial to return youth to 
an environment in which they met 
with failure and frustration? 

5. Experimental research should 
be conducted on the net effects of 
job search assistance programs such 
as MYEC. Are employment slots 
filled more quickly, or with more 
productive personnel? Or do employ- 
ment assistance projects simply 
rearrange the employment-seeker 
queue? What happens to the persons 
that would have landed the jobs? Is 
there any net benefit to society 
during a recession when there is a 
large labor surplus, and competition 
for jobs is fierce? 

Epilogue 

MYEC and the JVS project in gen- 
eral were exemplary in their will- 



ingness to subject themselves to 
rigorous evaluation. Evaluation is 
risky — it exposes oneself to the 
verification of failure. But this 
risk is mitigated when evaluation 
results are used to produce program 
improvement , and an ul timately 
satisfactory evaluation. This is 
the essence of the Program Develop- 
ment Evaluation process . It is 
unfortunate that the present report 
is a summative evaluation — it would 
have been better construed forma- 
tively . 

It is much to ask of a newly con- 
ceived program that it demonstrate 
^effectiveness in its first or second 
year of operation. Even private 
enterprise ventures routinely oper- 
ate at a loss early in their his- 
tory; this is considered normal, and 
unnoteworthy . Yet social service 
programs , addressing large problems 
in small ways, are expected to 
demonstrate efficacy almost immedi-^ 
ately. There can be no place for 
sloth or useless projects; there can 
also be no place for pie-in-the-sky 
expectations for instant success in 
dif f icul t areas . What is needed is 
systematic attention to the improve- 
ment of programs over time. Our 
na ti on mus t have the co ur age and 
commitment to provide the support 
that will allow program development 
to take place in a spiral of 
improvement over time . Pro j ect s to 
design, build, and test ballistic 
missle systems can take decades — why 
would less time be required to pre- 
vent youth crime? 
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Table 1 



Post-Randomization Checks on the Equivalency of the MYEC 
Treatment and Control Groups, 19 82 



MYEC 



J 



M 



SD 



Control 



M 



SD 



Age (years) 
Sex (male=l) 



16.81 .62 247 16.70 .72 149 .10 



.67 



— 252 



.63 ~ 154 .30 



Month enrolled 
in program 

Days from enrollment 
to follow-up 

Interpersonal 
Competency 

Psychological 
Health 



3.90 1.58 252 3.98 1.52 154 .60 



219.79 34.94 184 215.63 44.00 98 .39 



.89 .14 240 



.73 .21 245 



.88 .15 144 .58 



.70 .22 145 .21 



Rebellious 
Autonomy 



.61 .26 240 



.69 .24 145 .006* 



^The first month of the experimental evaluation (August, 1981) is coded 
"1," the second month (September) is coded "2," etc. 

* Statistically significant. 
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Table 2 



Mean Post-Test Psychosocial Development Scores 
for TiEC Treatment and Control Groups, 1982 



MYEC Con to 1 





H 


SD 


N 


M 


SD 


N 


P 


Interpersonal Competency^ 


.01 


.19 


176 


-.02 


.19 


101 


.65 


Psychological Health^ 


.02 


•?3 


180 


.05 


.26 


101 


.26 


Rebellious Autonomy^ 


.02 


.30 


176 


-.06 


.28 


102 


.27 


Attitude toward 

Education^^ 2 


.79 


.70 


185 


2.75 


.71 


108 


.95 


Average change from pretest 


to posttest 












^Average rating for each item 


; scale for 


items 


ran from 1 - 


5. 




c • . 

This IS an after-only score, 


ie. , 


no pretest 


is avail 


able . 







Table 3 



Percentage of MYEC Clients and Randomly Equivalent 
Controls Reporting Favorable Employment 
and Education Outcomes 



Outcome 


MYEC 


Controls 


P 


Employed 


\$ 

18 


21 


.68 


In school 


17 


26 


.10 


Favorable outcome 


36 


47 


.07 


(N) 


(172) 


(106) 





Note , Source is response to the question, "What 
are you doing mostly?" Response rates to this 
item are nearly equal for the two groups: 68% for 
MYEC, 69% for the control group. Favorable out- 
come means the respondent was employed or enrolled 
in school (or both). 
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himmi9^9^ ^ KTCC CllMtt Md Mudoaly Equivftleet 
tmtw^t^ 94ffmti»$ Variout Currtot Activities 





NTEC 


CoDCrols 


P 




(166) 


30 
(90) 


.27 


«Mir»«>t 


8 

(159) 


11 

(85) 


.57 


•r «Mipl«l»4l C*C.P. 


9 

(196) 


9 

(114) 


.99 


Att4MMliii| • CCP pr<itm 


J5 

(163) 


39 

(88) 


.04 




5 

(157) 


14 

(81) 


.02 




1 

(155) 


10 
(81) 


.002 




69 

(175) 


57 
(97) 


,05 



i»ii>»«»«r tnm immmmmmm m mm m 



tmit^ 9f m§l f«rctiitaf«t ar* rc*poniti to 
« iMiUiplt-<»HiM f«««Uii» '*W»t ar« ywi dpiogT" 
tMi!«>Mlffit« «mM Mrk at aany alttrDativaa as 
^llfNl. In a atNcaU ^acioo raspoadants vara 
•ilMNi acteMl tradMCion. N'a diffar 

itm ^atlM to ^atiM Neauaa not all raapon- 
4mf tflUmi tk9 toacrMCfiien to aark ooa raa- 
fmm for oacb lioo U tlw mlti-optioB quaation. 
f«f««tliotoa atiov tiio otNibor of raapondtnts to aaeh 
hmi tot«i S for WW • 252, total N for control 



534- 

423 



Table 5 



Treatment and Control Group Comparisons on Employment 
Outcomes and Self-Reported Arrests, 1982 



MYEC Control 

M SD N M SD 



Current hours worked 3.00 9.89 239 3.84 12.24 146 .46 

per week 

Current weekly wage 11.54 33.72 187 17.00 43.78 109 .23 

Total earnings last 174.58 592.38 189 160.76 509.05 112 .84 
six months 

Hours worked full-time 1.31 4.73 202 - 1.27.. 4.12. 117 ^ _ .95^ 
last six months 

Hours worked part-time 1.92 6.34 202 1.77 5.18 116 .83 

last six months 

Current hourly wage .58 1.50 192 .67 1.39 114 .57 

Transformed current^ .58 1.49 187 .72 1.69 109 .47 

hourly wage 

Transformed wages^ .75 2.22 189 .92 2.36 112 .55 

last six months 

Number of times picked .27 .69 185 .39 .89 109 .18 

up by police, 
last six months 



^Because of extreme skewness in the earnings data, the hourly and total wage 
variables were subjected to a logarithmic transformation. 
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Table 6 



IIYEC Progrsm Implementation Measures, August, 1982 
to February, 1982 



Implenentation 
measure 












Received no 


treatment 


Mean 


Median 


Mode 


SD 


Range 


N 


% 


Counseling contacts^ 


4.47 


3.54 


O.fl 


4.59 


0-31 


48 


19 


Educational services 
contact 


3.43 


0.20 


0.0 


8.96 


0-60 


171 


68 


Job referrals 


1.58 


0.43 


0.0 


2.69 


0-21 


129 


51 


Job placements 


0.30 


0.16 


0.0 


0,59 


0-3 


182 


72 



Note , N=252 for all measures. 
Includes personal meetings, and phone calls 
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Table 7 



Direct Contributions and Incremental Validity of a MYEC 
Implementation Measure to the Explanation of Work Outcomes 
According to Parsimonious Causal Models 







Increment 


p for 


Predictor 


beta 




to R 


increment 


Hours currently working per week (N = 198) 


Ethnicity (white = 1) 


.19 




.03 


.009* 


Age^ 


.14 




.02 


.05* 


MYEC participation 


.16 




.03 


.02* 


Weeks worked full-time 


in last six 


months (N = 


169) • 


Ethnicity (white = 1) 


.24 




.06 


.001* 


Month enrolled in 


-.21 




.04 


.006* 


program 


.2'0 








Age 




.03 


.01* 


MYEC participation 


.02 




.00 


.75 


Waeks worked part-time 


in last six 


months (N = 


169) 


Interpersonal Competency 


-.31 




.10 


.001* 


Month enrolled in program 


-.15 




.03 


.04* 


MYEC participation 


.15 




.02 


.04* 


Log-transformed 


current wages (N 


= 155) 




Ethnicity (white = 1) 


.17 




.03 


.03* 


MYEC participation 


.23 




.05 


.03* 


Log-transformed total earnings in last 


six 


months 


(N = 157) 


Interpersonal Competency 


-.23 




.05 


.003* 


Ethnicity (white = D 


.21 




.04 


.008* 


MYEC participation 


.16 




.02 


.04* 



Note , The components of the overall MYEC intervention were implemented to 
different degrees for different members of the treatment group. The incre 
ment to for MYEC participation shows the proportion of the variance in 
each outcome accounted for by a composite measure of participation in vari 
ous MYEC treatment components. MYEC was more fully implemented for stu- 
dents participating more fully in the various interventions. A positive 
beta for MYEC participation implies that persons who participated more 
fully in MYEC had higher scores on the particular outcome measure in ques- 
tion, net of measured pre-existing personal characteristics. Variables 
listed in each set above MYEC participation are those pre-test measures 
that significantly contributed to the prediction of each outcome. 

* Statistically significant. 
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Table 8 



Direct Contributions and Incremental Validity of a MYEC 
Implementation Measure to the Explanation of Psychosocial Outcomes 
and Arrests According to Parsimonious Causal Models 



Predictor beta • 


Xilu IT cute 11 L 

to R^ 


p for 
increment 


c oy \»n\j 1. iLCdxuii \ i « 


157) 




Psvcho lop ical hPrilth .^7 
Basic skills .20 
MYEC participation .10 


.22 
.04 
.01 


.001** 
.006** 
.14 


Interpersonal Competency (N 


= 156) 




Interpersonal Competency .18 
MYEC participation -.04 


.03 
.00 


.02** 
.58 


Rebellious Autonomy (N = 


155) 




Rebellious Autonomy .40 
MYEC participation -.13 


.16 
.01 


.001** 
.08* 


Re-enrollment in formal schooling (N = 141) 


Psychological Health .24 
MYEC participation .16 


.06 
.03 


.005** 
.003** 


Times arrested last six months 


(N = 158) 




Gender (female = 1) -.20 
MYEC participation -.12 


.04 
.01 


.01** 
.13 


Attitude towards Education (N = 156) 


Psychological health .20 
Ethnicity (white = 1) -.17 
Basic skills .16 
MYEC participation .13 


.04 
.03 
.03 
.01 


.01** 
.03** 
.04** 
.10* 



Note . The increment to R for MYEC participation shows the proportion of 
the variance in each outcome accounted for by a composite measure of parti 
cipation in various MYEC treatment components. A positive beta for MYEC 
participation implies that persons who participated more fully in MYEC had 
higher scores on the particular outcome measure in question, net of mea- 
sured pre-existing personal characteristics. Variables listed in each set 
above ^IYEC participation are those pre-test measures that significantly 
contributed to the prediction of each outcome. 

* Strong trend. 
** Statistically significant. 
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Table 9 



Direct Contributions and Incremental Validity of Participation 
in ^^YEC Treatment Components to the Explanation of Work Outcomes 
Controlling for Client Background and Pretest Measures 



Predictor beta 


Increment 
to 


p for 
increment 


Hours currently working per week (N = 198) 


Counseling .06 
Educational services .08 
Job referral .00 
Job placement .24 


.00 
.00 
.00 
.05 


.44 
.25 
.99 

.001** 


Weeks worked full-time in last six 


months (N = 


169) 


Counseling .04 
Educational services -.04 
Job referral .Oi 
Job placement .09 


.00 
.00 
.00 
.00 


.91 
.61 
.65 
.23 


Weeks worked part-time in last six 


months (N = 


169) 


Counseling -.13 
Educational services .34 
Job referral .01 
Job placement .11 


.01 
.12 
.00 
.01 


.09* 
.001** 
.94 
.13 


Log-transformed current wages (N = 155) 


Counseling -.06 
Educational services .20 
Job referral -.02 
Job placement .28 


.00 
.04 
.00 
.08 


.48 

.01** 

.83 

.001** 


Log-transformed total earnings in last 


six months 


(N = 157) 


Counseling -.09 
Educational services .22 
Job referral .13 
Job placement .15 


.00 
.05 
.01 
.02 


.31 

.004** 

.08* 

.07* 



Note , The increment to R for participation in a MYEC component shows the 
proportion of the variance in an outcome accounted for by participation in 
that component. A positive beta implies that persons who participated in 
that component had higher scores on the particular outcome measure in ques- 
tion, net of measured pre-existing personal characteristics. 

* Strong trend. 
** Statistically significant. 
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Table 10 



Dix-ect Contributions and Increnental Validity of Participation 
in MYEC Treatment Components to the Explanation of Psychosocial 
Outcomes and Arrest Controlling for Background and Pretest Measures 







Increment 


p for 


Predictor 


beta 


to R^ 


increment 


Psychological Health (N = 


157) 






-.05 


.00 


.51 




.05 


.00 


.49 


Job referral 


-.01 


.00 


-.93 


Job placement 


.16 


.03 


.02* 


Interpersonal 


Competency (N 


= 156) 






-.18 


.03 


.02* 




.02 


.00 


.83 


Job referral 


.09 


.00 


.31 


Job placement 


.05 


.00 


.55 


Rebellious 


Autonomy (N = 


155) 




Counseling 


-.07 


.00 


.38 


Educational Services 


-.06 


.00 


.44 


Job referral 


-.02 


.00 


.80 


Job placement 


-.10 


.00 


.18 


Re-enrollment, in 


formal schooling (N = 141) 




Counsel ing 


.12 


.01 


.18 


Educational Services 


.23 


.05 


.001* 


Job referral 


.10 


.01 


.22 


Job placement 1 


.11 


.01 


.19 


Times arrested last six months 


(N = 158) 




Counseling 


.01 


.00 


.90 


Educational Services 


-.08 


.00 


.33 


Job referral 


-.03 


.00 


.74 


Job placement 


-.13 


.01 


.10* 


Attitude towards Education (N = 156) 


Counseling 


-.13 


.01 


.12 


Educational Services 


.07 


.00 


.39 


Job referral 


-.09 


.00 


.34 


Job placement 


.17 


.03 


.03* 



Note , The increment to R for participation in a MYEC component shows the 
proportion of the variance in an outcome accounted for by participation in 
that component. A positive beta Implies that persons who participated in that 
component had higher scores on the particular outcome measure in question, net 
of measured pre-existing personal characteristics. The results for Attitude 
toward Education must be interpreted with special caution because this atti- 
tude was not measured at pretest. 

* Statistically significant. _c;An- AOCi 



Table 11 



Correlations between Extent of Participation in 
MYEC Project Components and Ethnicity (White = 1) 

(N = 238) 



Treatment component 




P 


Counseling contacts 


-.12 


.04* 


Educational services contacts 


-.11 


.04* 


Job referrals 


•01 


. ..43 


Job placements 


.16 


.006* 



* Statistically significant. 
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Table 12 



Direct Contributions and Incremental Validity of Explanatory 
Variables in a Model of Psychological Health 
at Fol low-Up 



Predictor 


r 


beta 


Increment 
to R 


p for 
increment 


Statistically si 


gnif icant pr^d: 


LCtors 




Psychological Health (pre) 


.41 


.31 


.17 


.001* 


Basic Skills Pretest 


.26 


.23' 


.04 


.01* 


Rebellious Autonomy (pre) 


-.21 


-.20 


.03 


.02* 


Participation in formal 
schooling (concurrent ) 


-.23 


-.17 


.03 


.04* 


Concurrent variables 


not significant net of 


pre-test measures 


Hours per week currently 
working 


.10 


.07 


.00 


.37 


Weeks worked full-time 
last six months 


.05 


.04 


.00 


.61 


Weeks worked part-time 
last six months 


-.01 


.01 


.00 


.85 


Logged current v/ages 


.08 


.08 


.01 


.28 


Logged total earnings 


-.03 


-.04 


.00 


.58 



last six months 



Note . The model was constructed by allowing significant pre-test or 
background predictors of Psychological Health at follow-up to enter the 
equation first, and then entering each of the concurrent work and 
schooling variables into the equation. Only participation in formal 
schooling significantly adds to the explanation of Psychological Health 
when pre-test measures are controlled for. Due to differential patterns 
of missing data, the models used to estimate the regression parameters 
for the non-significant regressors vary slightly from those for which 
the parameters are shown here. The differences are trivial. N's range 
from 132 to 156. 



* Statistically significant. 
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Table 13 



Direct Contributions and Incremental Validity of Explanatory 
Variables in a Model of Interpersonal Competency 
at Follow-Up 



Increment p for 

Predictor r beta to R increment 

Statistically significant predictors 



Interpersonal Competency .21 .21 .05 ' .01* 

(Pre-test) 

Participation in formal -.25 -.24 .06 .005* 

schooling (concurrent ) ^ 



Concurrent variables not significant net of pre-test measures 



Hours per week currently .09 .09 .01 .27 

v7orking" 

Weeks worked full-time .01 .03 .00 .75 

last six months 

Weeks worked part-time -.07 -.01 .00 .92 

last six months 

Logged current wages .04 .07 .01 .38 

i 

Logged total earnings .06 .11 .01 .19 

last six months 



Note. The model was constructed by allowing significant pre-test or 
background predictors of Interpersonal Competency at follow-up to enter 
the equation first, and then entering each of the concurrent work and 
schooling variables into the equation. Only participation in formal 
schooling significantly adds to the explanation of Interpersonal Compe- 
tency when pre-test measures are controlled for. Due to differential 
patterns of missing data, the models used to estimate the regression 
parameters for the non-significant regressors vary slightly from those 
for which the parameters are shown here. The differences are trivial. 
N's range from 131 to 155. 

* Statistically significant. 
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Table 14 



Direct Contributions and Incremental Validity of Explanatory 
Variables in a Model of Rebellious Autonomy 
at Fol low-Up 



Predictor 



r beta 
Statistically significant predictors 



Increment 
to 



p for 
increment 



Rebellious Autonomy (pre) .40 

Weeks worked full-time .15 
last six months 



.40 
.15 



.16 
.02 



.001* 
.04* 



Concurrent variables not significant net of pre-test measures 



Hours per v;eek currently .14 
working 

Weeks worked part-time .01 
last six months 

Logged current wages .09 

Logged total earnings .16 
last six months 

Participation in formal .08 
schooling 



.12 

.01 

.08 
.12 

.12 



.01 

.00 

.01 
.01 

.01 



.17 

.85 

.33 
.12 

.15 



Note . The model was constructed by allowing significant pre-test or 
background predictors of Rebellious Autonomy at follow-up to enter the 
equation first, and then entering each of the concurrent work and 
schooling variables into the equation. Only participation in formal 
schooling significantly adds to the explanation of Rebellious Autonomy 
when pre-test measures are controlled for. Due to differential patterns 
of missing data, the models used to estimate the regression parameters 
for the non-significant regressors vary slightly from those for which 
the parameters are shown here. Tlie differences are trivial. N's range 
from 130 to 154. 



* Statistically significant. 
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Table 15 



Correlations between Personal Characteristics Measured 
at Pre-test and Follov7-up with Number of Self-Reported 

Arrests 



Personal characteristic r N 



Measured at pretest 

Age -.13* 285. 

Sex (female = 1) ■ . -.22** 288 

Ethnicity (white = 1) .03 291 

Month enrolled in program .00 294 

Basic skills test -.01 179 

Psychological Health .00 283 

Rebellious Autonomy -.02 284 

Interpersonal Competency -.04 280 

Extent of participation in MEC -.12* 287 

Measured at follow-up 

Hours worked in past week .11 273 

Full-time employment .04 292 

Part-time employment -.01 291 

Log-transformed current wages .00 269 

Log-transformed total .01 274 

earnings last six months 

Re-enrollment in school -.08 259 

Psychological health -.10 280 

Rebellious Autonomy -.04 277 

Interpersonal Competency -.14* 278 
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Appendix A 

Items in Scales Used to Measure Psychosocial Developnent 
in the MYEC Evaluation 

Psychologic al Health 

Sco ring Item 

- Others see me as a loser. 

- It's pretty tough to be me. 

- No matter what I do, it's not going to make any difference. 

- I feel I do not have much to be proud of. 

- These days I feel I'm just not a part of things. 

- I don't know whom I can count on these days. 

- I have little influence over what happens to me. 
■* Good luck is more important than hard work. 

Pretest alpha = .61 
Posttest alpha = .59 
Eetest reliability = .40 



Int erp ersonal Competency 

S coring Item 

+ I knov; how to get along with adults. 

+ I am the kind of person who can make it if I try. 

+ If I want to, I can explain things well. 

+ Others see me as successful. 

+ I like myself. 

+ My friends regard me as a person with good sense. 

- I feel no one really cares what happens to me. 

Pretest alpha = .45 
Posttest alpha = .46 
Retest reliability = .19 



Rebel lious Autonomy 

Sc oring Item 

+ What I do with my time is my own business. 

+ I don't like anybody telling me what to do. 

+ I can get along just fine on my own. 

+ What happens to me is my own doing. 

+ Nobody has the right to tell me how to spend my money. 

Pretest alpha = .49 
Posttest alpha = .49 
Retest reliability = .39 
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